2024-09-16T09:40:36.662 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T09:40:36.677 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T09:40:36.759 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/avan-2024-09-16_08:23:53-orch:cephadm-wip-athakkar-testing-2024-09-12-1857-distro-default-smithi/7907075 branch: wip-athakkar-testing-2024-09-12-1857 description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} email: athakkar@redhat.com first_in_suite: false job_id: '7907075' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: avan-2024-09-16_08:23:53-orch:cephadm-wip-athakkar-testing-2024-09-12-1857-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-athakkar-testing-2024-09-12-1857 ceph: conf: global: mon election default strategy: 3 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(PG_ - \(OSD_ - \(OBJECT_ - \(POOL_APP_NOT_ENABLED\) log-only-match: - CEPHADM_ sha1: 66e33d6af1496291b0602ee0a859e68145f17dad ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: extra_system_packages: deb: - python3-pytest rpm: - python3-pytest flavor: default sha1: 66e33d6af1496291b0602ee0a859e68145f17dad selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-athakkar-testing-2024-09-12-1857 sha1: 66e33d6af1496291b0602ee0a859e68145f17dad owner: scheduled_avan@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 4787 sha1: 66e33d6af1496291b0602ee0a859e68145f17dad sleep_before_teardown: 0 subset: 1/20 suite: orch:cephadm suite_branch: wip-athakkar-testing-2024-09-12-1857 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_66e33d6af1496291b0602ee0a859e68145f17dad/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 66e33d6af1496291b0602ee0a859e68145f17dad targets: smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKLe2HhlDiboBG3uD2qGaP7gecZV8bKEv5z1tQ05u6W5f+kJLo1HVdqo65LL26+pFmu2hxr23IZ6wbx0Hxii8eM= smithi179.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJr8ibaO8RVHJTY7zU64JPMF1SZ6KOoX3GXl43GZJb333fpNlrgeedumsX7/QSE3Vgn3BtTA8fWSOhKxrOgPh+A= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test_python.sh timeout: 1h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_08:23:53 tube: smithi user: avan verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T09:40:36.760 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_66e33d6af1496291b0602ee0a859e68145f17dad/qa; will attempt to use it 2024-09-16T09:40:36.760 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_66e33d6af1496291b0602ee0a859e68145f17dad/qa/tasks 2024-09-16T09:40:36.761 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T09:40:36.762 INFO:teuthology.task.internal:Checking packages... 2024-09-16T09:40:36.784 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '66e33d6af1496291b0602ee0a859e68145f17dad' 2024-09-16T09:40:36.784 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T09:40:36.784 INFO:teuthology.packaging:ref: None 2024-09-16T09:40:36.784 INFO:teuthology.packaging:tag: None 2024-09-16T09:40:36.784 INFO:teuthology.packaging:branch: wip-athakkar-testing-2024-09-12-1857 2024-09-16T09:40:36.784 INFO:teuthology.packaging:sha1: 66e33d6af1496291b0602ee0a859e68145f17dad 2024-09-16T09:40:36.785 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-athakkar-testing-2024-09-12-1857 2024-09-16T09:40:36.929 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4882.g66e33d6a 2024-09-16T09:40:36.930 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T09:40:36.939 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T09:40:36.939 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T09:40:36.948 INFO:teuthology.task.internal:Saving configuration 2024-09-16T09:40:36.959 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T09:40:36.968 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T09:40:36.993 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/avan-2024-09-16_08:23:53-orch:cephadm-wip-athakkar-testing-2024-09-12-1857-distro-default-smithi/7907075', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 09:31:53.252519', 'locked_by': 'scheduled_avan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKLe2HhlDiboBG3uD2qGaP7gecZV8bKEv5z1tQ05u6W5f+kJLo1HVdqo65LL26+pFmu2hxr23IZ6wbx0Hxii8eM='} 2024-09-16T09:40:37.011 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/avan-2024-09-16_08:23:53-orch:cephadm-wip-athakkar-testing-2024-09-12-1857-distro-default-smithi/7907075', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 09:31:53.254462', 'locked_by': 'scheduled_avan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNCPLW3zeRuuV6Ollifb77VItVZGOnbIawcyKJoxXb/UjElpzHQxWUyi0+KSfwiGYaoAOcrlFRI7NfSVxdkWnI='} 2024-09-16T09:40:37.011 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T09:40:37.019 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-16T09:40:37.019 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-16T09:40:37.019 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T09:40:37.085 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f74446f3ac0>, signals=[15]) 2024-09-16T09:40:37.085 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T09:40:37.091 INFO:teuthology.task.internal:Opening connections... 2024-09-16T09:40:37.092 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2024-09-16T09:40:37.093 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T09:40:37.169 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi179.front.sepia.ceph.com 2024-09-16T09:40:37.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T09:40:37.188 ERROR:teuthology.orchestra.connection:Error authenticating with smithi179.front.sepia.ceph.com: BadHostKeyException('smithi179.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:Otaz/sQ3vUdDDYUOLG5FcoqVy8DfW0psSMM6BwlphLY), PKey(alg=ECDSA, bits=256, fp=SHA256:v/eNhtyghd7jjnWzGKQSq6aSpnQEt4X9Yv7vOdvvl30)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/connection.py", line 111, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi179.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNCPLW3zeRuuV6Ollifb77VItVZGOnbIawcyKJoxXb/UjElpzHQxWUyi0+KSfwiGYaoAOcrlFRI7NfSVxdkWnI=', expected 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJr8ibaO8RVHJTY7zU64JPMF1SZ6KOoX3GXl43GZJb333fpNlrgeedumsX7/QSE3Vgn3BtTA8fWSOhKxrOgPh+A=' 2024-09-16T09:40:41.206 ERROR:teuthology.orchestra.connection:Error authenticating with smithi179.front.sepia.ceph.com: BadHostKeyException('smithi179.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:Otaz/sQ3vUdDDYUOLG5FcoqVy8DfW0psSMM6BwlphLY), PKey(alg=ECDSA, bits=256, fp=SHA256:v/eNhtyghd7jjnWzGKQSq6aSpnQEt4X9Yv7vOdvvl30)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/connection.py", line 111, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi179.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNCPLW3zeRuuV6Ollifb77VItVZGOnbIawcyKJoxXb/UjElpzHQxWUyi0+KSfwiGYaoAOcrlFRI7NfSVxdkWnI=', expected 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJr8ibaO8RVHJTY7zU64JPMF1SZ6KOoX3GXl43GZJb333fpNlrgeedumsX7/QSE3Vgn3BtTA8fWSOhKxrOgPh+A=' 2024-09-16T09:40:48.226 ERROR:teuthology.orchestra.connection:Error authenticating with smithi179.front.sepia.ceph.com: BadHostKeyException('smithi179.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:Otaz/sQ3vUdDDYUOLG5FcoqVy8DfW0psSMM6BwlphLY), PKey(alg=ECDSA, bits=256, fp=SHA256:v/eNhtyghd7jjnWzGKQSq6aSpnQEt4X9Yv7vOdvvl30)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/connection.py", line 111, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi179.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNCPLW3zeRuuV6Ollifb77VItVZGOnbIawcyKJoxXb/UjElpzHQxWUyi0+KSfwiGYaoAOcrlFRI7NfSVxdkWnI=', expected 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJr8ibaO8RVHJTY7zU64JPMF1SZ6KOoX3GXl43GZJb333fpNlrgeedumsX7/QSE3Vgn3BtTA8fWSOhKxrOgPh+A=' 2024-09-16T09:40:58.245 ERROR:teuthology.orchestra.connection:Error authenticating with smithi179.front.sepia.ceph.com: BadHostKeyException('smithi179.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:Otaz/sQ3vUdDDYUOLG5FcoqVy8DfW0psSMM6BwlphLY), PKey(alg=ECDSA, bits=256, fp=SHA256:v/eNhtyghd7jjnWzGKQSq6aSpnQEt4X9Yv7vOdvvl30)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/connection.py", line 111, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi179.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNCPLW3zeRuuV6Ollifb77VItVZGOnbIawcyKJoxXb/UjElpzHQxWUyi0+KSfwiGYaoAOcrlFRI7NfSVxdkWnI=', expected 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJr8ibaO8RVHJTY7zU64JPMF1SZ6KOoX3GXl43GZJb333fpNlrgeedumsX7/QSE3Vgn3BtTA8fWSOhKxrOgPh+A=' 2024-09-16T09:41:11.265 ERROR:teuthology.orchestra.connection:Error authenticating with smithi179.front.sepia.ceph.com: BadHostKeyException('smithi179.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:Otaz/sQ3vUdDDYUOLG5FcoqVy8DfW0psSMM6BwlphLY), PKey(alg=ECDSA, bits=256, fp=SHA256:v/eNhtyghd7jjnWzGKQSq6aSpnQEt4X9Yv7vOdvvl30)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/connection.py", line 111, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi179.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNCPLW3zeRuuV6Ollifb77VItVZGOnbIawcyKJoxXb/UjElpzHQxWUyi0+KSfwiGYaoAOcrlFRI7NfSVxdkWnI=', expected 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJr8ibaO8RVHJTY7zU64JPMF1SZ6KOoX3GXl43GZJb333fpNlrgeedumsX7/QSE3Vgn3BtTA8fWSOhKxrOgPh+A=' 2024-09-16T09:41:27.285 ERROR:teuthology.orchestra.connection:Error authenticating with smithi179.front.sepia.ceph.com: BadHostKeyException('smithi179.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:Otaz/sQ3vUdDDYUOLG5FcoqVy8DfW0psSMM6BwlphLY), PKey(alg=ECDSA, bits=256, fp=SHA256:v/eNhtyghd7jjnWzGKQSq6aSpnQEt4X9Yv7vOdvvl30)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/connection.py", line 111, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi179.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNCPLW3zeRuuV6Ollifb77VItVZGOnbIawcyKJoxXb/UjElpzHQxWUyi0+KSfwiGYaoAOcrlFRI7NfSVxdkWnI=', expected 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJr8ibaO8RVHJTY7zU64JPMF1SZ6KOoX3GXl43GZJb333fpNlrgeedumsX7/QSE3Vgn3BtTA8fWSOhKxrOgPh+A=' 2024-09-16T09:41:46.305 ERROR:teuthology.orchestra.connection:Error authenticating with smithi179.front.sepia.ceph.com: BadHostKeyException('smithi179.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:Otaz/sQ3vUdDDYUOLG5FcoqVy8DfW0psSMM6BwlphLY), PKey(alg=ECDSA, bits=256, fp=SHA256:v/eNhtyghd7jjnWzGKQSq6aSpnQEt4X9Yv7vOdvvl30)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/connection.py", line 111, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi179.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNCPLW3zeRuuV6Ollifb77VItVZGOnbIawcyKJoxXb/UjElpzHQxWUyi0+KSfwiGYaoAOcrlFRI7NfSVxdkWnI=', expected 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJr8ibaO8RVHJTY7zU64JPMF1SZ6KOoX3GXl43GZJb333fpNlrgeedumsX7/QSE3Vgn3BtTA8fWSOhKxrOgPh+A=' 2024-09-16T09:42:08.325 ERROR:teuthology.orchestra.connection:Error authenticating with smithi179.front.sepia.ceph.com: BadHostKeyException('smithi179.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:Otaz/sQ3vUdDDYUOLG5FcoqVy8DfW0psSMM6BwlphLY), PKey(alg=ECDSA, bits=256, fp=SHA256:v/eNhtyghd7jjnWzGKQSq6aSpnQEt4X9Yv7vOdvvl30)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/connection.py", line 111, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi179.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNCPLW3zeRuuV6Ollifb77VItVZGOnbIawcyKJoxXb/UjElpzHQxWUyi0+KSfwiGYaoAOcrlFRI7NfSVxdkWnI=', expected 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJr8ibaO8RVHJTY7zU64JPMF1SZ6KOoX3GXl43GZJb333fpNlrgeedumsX7/QSE3Vgn3BtTA8fWSOhKxrOgPh+A=' 2024-09-16T09:42:33.345 ERROR:teuthology.orchestra.connection:Error authenticating with smithi179.front.sepia.ceph.com: BadHostKeyException('smithi179.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:Otaz/sQ3vUdDDYUOLG5FcoqVy8DfW0psSMM6BwlphLY), PKey(alg=ECDSA, bits=256, fp=SHA256:v/eNhtyghd7jjnWzGKQSq6aSpnQEt4X9Yv7vOdvvl30)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/connection.py", line 111, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi179.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNCPLW3zeRuuV6Ollifb77VItVZGOnbIawcyKJoxXb/UjElpzHQxWUyi0+KSfwiGYaoAOcrlFRI7NfSVxdkWnI=', expected 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJr8ibaO8RVHJTY7zU64JPMF1SZ6KOoX3GXl43GZJb333fpNlrgeedumsX7/QSE3Vgn3BtTA8fWSOhKxrOgPh+A=' 2024-09-16T09:43:01.365 ERROR:teuthology.orchestra.connection:Error authenticating with smithi179.front.sepia.ceph.com: BadHostKeyException('smithi179.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:Otaz/sQ3vUdDDYUOLG5FcoqVy8DfW0psSMM6BwlphLY), PKey(alg=ECDSA, bits=256, fp=SHA256:v/eNhtyghd7jjnWzGKQSq6aSpnQEt4X9Yv7vOdvvl30)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/connection.py", line 111, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi179.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNCPLW3zeRuuV6Ollifb77VItVZGOnbIawcyKJoxXb/UjElpzHQxWUyi0+KSfwiGYaoAOcrlFRI7NfSVxdkWnI=', expected 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJr8ibaO8RVHJTY7zU64JPMF1SZ6KOoX3GXl43GZJb333fpNlrgeedumsX7/QSE3Vgn3BtTA8fWSOhKxrOgPh+A=' 2024-09-16T09:43:01.366 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/internal/__init__.py", line 180, in connect rem.connect() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 387, in connect self.ssh = connection.connect(**args) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/connection.py", line 108, in connect while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'connect to smithi179.front.sepia.ceph.com' reached maximum tries (10) after waiting for 145 seconds 2024-09-16T09:43:01.545 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=654e245f0ca245708f6b7af986dd99c8 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/internal/__init__.py", line 180, in connect rem.connect() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 387, in connect self.ssh = connection.connect(**args) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/connection.py", line 108, in connect while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'connect to smithi179.front.sepia.ceph.com' reached maximum tries (10) after waiting for 145 seconds 2024-09-16T09:43:01.547 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T09:43:01.562 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} failure_reason: '''connect to smithi179.front.sepia.ceph.com'' reached maximum tries (10) after waiting for 145 seconds' owner: scheduled_avan@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=654e245f0ca245708f6b7af986dd99c8 status: fail success: false 2024-09-16T09:43:01.562 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T09:43:01.653 INFO:teuthology.run:FAIL