2023-12-13T20:42:01.712 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-13T20:42:01.713 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-13T20:42:01.845 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd2d9d70940>, signals=[15]) 2023-12-13T20:42:01.846 INFO:teuthology.lock.ops:Start node 'smithi006.front.sepia.ceph.com' reimaging 2023-12-13T20:42:01.846 INFO:teuthology.lock.ops:Updating [smithi006.front.sepia.ceph.com]: reset os type and version on server 2023-12-13T20:42:01.847 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2023-12-13T20:42:01.869 INFO:teuthology.lock.ops:Start node 'smithi177.front.sepia.ceph.com' reimaging 2023-12-13T20:42:01.870 INFO:teuthology.lock.ops:Updating [smithi177.front.sepia.ceph.com]: reset os type and version on server 2023-12-13T20:42:01.870 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2023-12-13T20:42:02.009 DEBUG:teuthology.provision.fog.smithi006:Requesting image smithi_rhel_8.4 (ID 7) 2023-12-13T20:42:02.015 DEBUG:teuthology.provision.fog.smithi177:Requesting image smithi_rhel_8.4 (ID 7) 2023-12-13T20:42:02.057 INFO:teuthology.provision.fog.smithi177:Scheduling deploy of rhel 8.4 2023-12-13T20:42:02.061 INFO:teuthology.provision.fog.smithi006:Scheduling deploy of rhel 8.4 2023-12-13T20:42:02.312 INFO:teuthology.orchestra.console.smithi177:Power off 2023-12-13T20:42:02.313 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-13T20:42:02.325 INFO:teuthology.orchestra.console.smithi006:Power off 2023-12-13T20:42:02.326 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-13T20:42:02.334 DEBUG:teuthology.orchestra.console.smithi177:power off output: Chassis Power Control: Down/Off 2023-12-13T20:42:02.334 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-13T20:42:02.348 DEBUG:teuthology.orchestra.console.smithi006:power off output: Chassis Power Control: Down/Off 2023-12-13T20:42:02.349 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-13T20:42:02.357 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2023-12-13T20:42:02.370 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2023-12-13T20:42:06.361 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-13T20:42:06.372 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-13T20:42:06.471 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2023-12-13T20:42:06.480 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2023-12-13T20:42:10.474 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-13T20:42:10.481 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-13T20:42:10.583 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off 2023-12-13T20:42:10.589 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is off 2023-12-13T20:42:10.684 INFO:teuthology.orchestra.console.smithi177:Power off completed 2023-12-13T20:42:10.690 INFO:teuthology.orchestra.console.smithi006:Power off completed 2023-12-13T20:42:10.786 INFO:teuthology.orchestra.console.smithi177:Power on 2023-12-13T20:42:10.786 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-13T20:42:10.793 INFO:teuthology.orchestra.console.smithi006:Power on 2023-12-13T20:42:10.793 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-13T20:42:10.807 DEBUG:teuthology.orchestra.console.smithi177:power on output: Chassis Power Control: Up/On 2023-12-13T20:42:10.807 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-13T20:42:10.816 DEBUG:teuthology.orchestra.console.smithi006:power on output: Chassis Power Control: Up/On 2023-12-13T20:42:10.816 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-13T20:42:10.829 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off 2023-12-13T20:42:10.837 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is off 2023-12-13T20:42:14.833 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-13T20:42:14.841 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-13T20:42:14.943 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off 2023-12-13T20:42:14.950 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2023-12-13T20:42:15.050 INFO:teuthology.orchestra.console.smithi006:Power on completed 2023-12-13T20:42:15.151 INFO:teuthology.provision.fog.smithi006:Waiting for deploy to finish 2023-12-13T20:42:18.945 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-13T20:42:19.055 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2023-12-13T20:42:19.156 INFO:teuthology.orchestra.console.smithi177:Power on completed 2023-12-13T20:42:19.256 INFO:teuthology.provision.fog.smithi177:Waiting for deploy to finish 2023-12-13T20:44:46.042 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-13T20:44:50.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-13T20:45:08.497 WARNING:teuthology.provision.fog.smithi177:[Errno None] Unable to connect to port 22 on 172.21.15.177 2023-12-13T20:45:14.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-13T20:45:17.553 WARNING:teuthology.provision.fog.smithi177:[Errno None] Unable to connect to port 22 on 172.21.15.177 2023-12-13T20:45:23.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-13T20:45:26.613 WARNING:teuthology.provision.fog.smithi177:[Errno None] Unable to connect to port 22 on 172.21.15.177 2023-12-13T20:45:32.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-13T20:45:35.697 WARNING:teuthology.provision.fog.smithi177:[Errno None] Unable to connect to port 22 on 172.21.15.177 2023-12-13T20:45:41.699 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-13T20:45:44.753 WARNING:teuthology.provision.fog.smithi177:[Errno None] Unable to connect to port 22 on 172.21.15.177 2023-12-13T20:45:50.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-13T20:46:16.182 ERROR:teuthology.orchestra.connection:Error authenticating with smithi006.front.sepia.ceph.com: AuthenticationException('Authentication timeout.') 2023-12-13T20:46:27.405 ERROR:teuthology.orchestra.connection:Error authenticating with smithi006.front.sepia.ceph.com: BadHostKeyException('smithi006.front.sepia.ceph.com', , ) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi006.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDgsOX29rBM7+YficMiXzjSgFj1F7RENnlJ9+LzzSvHWODpjkZ+ZBzagBUasy1VN5Hb9AiKbQCtoSiOEoYD4jNg=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIOYO0UwwFB9dm7zre9ci2RwYHlsRuJuAmp6SeipOqD4Q' 2023-12-13T20:46:34.424 ERROR:teuthology.orchestra.connection:Error authenticating with smithi006.front.sepia.ceph.com: BadHostKeyException('smithi006.front.sepia.ceph.com', , ) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi006.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDgsOX29rBM7+YficMiXzjSgFj1F7RENnlJ9+LzzSvHWODpjkZ+ZBzagBUasy1VN5Hb9AiKbQCtoSiOEoYD4jNg=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIOYO0UwwFB9dm7zre9ci2RwYHlsRuJuAmp6SeipOqD4Q' 2023-12-13T20:46:36.822 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-12-13T20:46:44.442 ERROR:teuthology.orchestra.connection:Error authenticating with smithi006.front.sepia.ceph.com: BadHostKeyException('smithi006.front.sepia.ceph.com', , ) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi006.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDgsOX29rBM7+YficMiXzjSgFj1F7RENnlJ9+LzzSvHWODpjkZ+ZBzagBUasy1VN5Hb9AiKbQCtoSiOEoYD4jNg=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIOYO0UwwFB9dm7zre9ci2RwYHlsRuJuAmp6SeipOqD4Q' 2023-12-13T20:46:50.757 WARNING:teuthology.provision.fog.smithi177:timed out 2023-12-13T20:46:56.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-13T20:46:57.069 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-13T20:46:57.069 DEBUG:teuthology.orchestra.run.smithi177:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-13T20:46:57.334 INFO:teuthology.provision.fog.smithi177:Node is ready 2023-12-13T20:46:57.335 DEBUG:teuthology.orchestra.run.smithi177:> hostname 2023-12-13T20:46:57.390 INFO:teuthology.orchestra.run.smithi177.stdout:smithi177.front.sepia.ceph.com 2023-12-13T20:46:57.391 DEBUG:teuthology.orchestra.run.smithi177:> grep smithi177.front.sepia.ceph.com /etc/hosts 2023-12-13T20:46:57.447 INFO:teuthology.orchestra.run.smithi177.stdout:172.21.15.177 smithi177.front.sepia.ceph.com smithi177 2023-12-13T20:46:57.447 DEBUG:teuthology.orchestra.run.smithi177:> sudo hostname smithi177 2023-12-13T20:46:57.459 ERROR:teuthology.orchestra.connection:Error authenticating with smithi006.front.sepia.ceph.com: BadHostKeyException('smithi006.front.sepia.ceph.com', , ) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi006.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDgsOX29rBM7+YficMiXzjSgFj1F7RENnlJ9+LzzSvHWODpjkZ+ZBzagBUasy1VN5Hb9AiKbQCtoSiOEoYD4jNg=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIOYO0UwwFB9dm7zre9ci2RwYHlsRuJuAmp6SeipOqD4Q' 2023-12-13T20:46:57.557 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i -e 's/smithi177.front.sepia.ceph.com/smithi177/g' /etc/hosts 2023-12-13T20:46:57.583 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i -e 's/172.21.15.177/172.21.15.177/g' /etc/hosts 2023-12-13T20:46:57.648 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i -e 's/smithi177.front.sepia.ceph.com/smithi177/g' /etc/hostname 2023-12-13T20:46:57.712 DEBUG:teuthology.orchestra.run.smithi177:> sudo hostname smithi177 2023-12-13T20:46:57.775 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2023-12-13T20:46:57.829 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="Red Hat Enterprise Linux" 2023-12-13T20:46:57.830 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="8.4 (Ootpa)" 2023-12-13T20:46:57.830 INFO:teuthology.orchestra.run.smithi177.stdout:ID="rhel" 2023-12-13T20:46:57.831 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="fedora" 2023-12-13T20:46:57.831 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="8.4" 2023-12-13T20:46:57.831 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el8" 2023-12-13T20:46:57.831 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-13T20:46:57.832 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2023-12-13T20:46:57.832 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-13T20:46:57.832 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://www.redhat.com/" 2023-12-13T20:46:57.833 INFO:teuthology.orchestra.run.smithi177.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-13T20:46:57.833 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-13T20:46:57.833 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-12-13T20:46:57.834 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-13T20:46:57.834 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-13T20:46:57.834 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-13T20:46:57.835 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-13T20:46:57.836 INFO:teuthology.provision.fog.smithi177:Deploy complete! 2023-12-13T20:46:57.837 DEBUG:teuthology.parallel:result is None 2023-12-13T20:47:13.478 ERROR:teuthology.orchestra.connection:Error authenticating with smithi006.front.sepia.ceph.com: BadHostKeyException('smithi006.front.sepia.ceph.com', , ) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi006.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDgsOX29rBM7+YficMiXzjSgFj1F7RENnlJ9+LzzSvHWODpjkZ+ZBzagBUasy1VN5Hb9AiKbQCtoSiOEoYD4jNg=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIOYO0UwwFB9dm7zre9ci2RwYHlsRuJuAmp6SeipOqD4Q' 2023-12-13T20:47:32.496 ERROR:teuthology.orchestra.connection:Error authenticating with smithi006.front.sepia.ceph.com: BadHostKeyException('smithi006.front.sepia.ceph.com', , ) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi006.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDgsOX29rBM7+YficMiXzjSgFj1F7RENnlJ9+LzzSvHWODpjkZ+ZBzagBUasy1VN5Hb9AiKbQCtoSiOEoYD4jNg=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIOYO0UwwFB9dm7zre9ci2RwYHlsRuJuAmp6SeipOqD4Q' 2023-12-13T20:47:54.514 ERROR:teuthology.orchestra.connection:Error authenticating with smithi006.front.sepia.ceph.com: BadHostKeyException('smithi006.front.sepia.ceph.com', , ) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi006.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDgsOX29rBM7+YficMiXzjSgFj1F7RENnlJ9+LzzSvHWODpjkZ+ZBzagBUasy1VN5Hb9AiKbQCtoSiOEoYD4jNg=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIOYO0UwwFB9dm7zre9ci2RwYHlsRuJuAmp6SeipOqD4Q' 2023-12-13T20:48:19.531 ERROR:teuthology.orchestra.connection:Error authenticating with smithi006.front.sepia.ceph.com: BadHostKeyException('smithi006.front.sepia.ceph.com', , ) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi006.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDgsOX29rBM7+YficMiXzjSgFj1F7RENnlJ9+LzzSvHWODpjkZ+ZBzagBUasy1VN5Hb9AiKbQCtoSiOEoYD4jNg=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIOYO0UwwFB9dm7zre9ci2RwYHlsRuJuAmp6SeipOqD4Q' 2023-12-13T20:48:47.548 ERROR:teuthology.orchestra.connection:Error authenticating with smithi006.front.sepia.ceph.com: BadHostKeyException('smithi006.front.sepia.ceph.com', , ) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi006.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDgsOX29rBM7+YficMiXzjSgFj1F7RENnlJ9+LzzSvHWODpjkZ+ZBzagBUasy1VN5Hb9AiKbQCtoSiOEoYD4jNg=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIOYO0UwwFB9dm7zre9ci2RwYHlsRuJuAmp6SeipOqD4Q' 2023-12-13T20:48:47.549 WARNING:teuthology.provision.fog.smithi006:'connect to smithi006.front.sepia.ceph.com' reached maximum tries (11) after waiting for 145 seconds 2023-12-13T20:48:53.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-13T20:48:53.794 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-13T20:48:53.851 DEBUG:teuthology.orchestra.run.smithi006:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-13T20:48:54.046 INFO:teuthology.provision.fog.smithi006:Node is ready 2023-12-13T20:48:54.046 DEBUG:teuthology.orchestra.run.smithi006:> hostname 2023-12-13T20:48:54.061 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006.front.sepia.ceph.com 2023-12-13T20:48:54.061 DEBUG:teuthology.orchestra.run.smithi006:> grep smithi006.front.sepia.ceph.com /etc/hosts 2023-12-13T20:48:54.117 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6 smithi006.front.sepia.ceph.com smithi006 2023-12-13T20:48:54.118 DEBUG:teuthology.orchestra.run.smithi006:> sudo hostname smithi006 2023-12-13T20:48:54.229 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i -e 's/smithi006.front.sepia.ceph.com/smithi006/g' /etc/hosts 2023-12-13T20:48:54.257 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i -e 's/172.21.15.6/172.21.15.6/g' /etc/hosts 2023-12-13T20:48:54.322 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i -e 's/smithi006.front.sepia.ceph.com/smithi006/g' /etc/hostname 2023-12-13T20:48:54.387 DEBUG:teuthology.orchestra.run.smithi006:> sudo hostname smithi006 2023-12-13T20:48:54.452 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release 2023-12-13T20:48:54.506 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="Red Hat Enterprise Linux" 2023-12-13T20:48:54.506 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="8.4 (Ootpa)" 2023-12-13T20:48:54.507 INFO:teuthology.orchestra.run.smithi006.stdout:ID="rhel" 2023-12-13T20:48:54.507 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="fedora" 2023-12-13T20:48:54.507 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="8.4" 2023-12-13T20:48:54.507 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el8" 2023-12-13T20:48:54.507 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-13T20:48:54.507 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31" 2023-12-13T20:48:54.507 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-13T20:48:54.507 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://www.redhat.com/" 2023-12-13T20:48:54.507 INFO:teuthology.orchestra.run.smithi006.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-13T20:48:54.507 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-13T20:48:54.507 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-12-13T20:48:54.507 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-13T20:48:54.507 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-13T20:48:54.508 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-13T20:48:54.508 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-13T20:48:54.508 INFO:teuthology.provision.fog.smithi006:Deploy complete! 2023-12-13T20:48:54.509 DEBUG:teuthology.parallel:result is None 2023-12-13T20:48:54.658 INFO:teuthology.lock.ops:Checking smithi006.front.sepia.ceph.com 2023-12-13T20:48:54.659 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-13T20:48:54.685 INFO:teuthology.lock.ops:Checking smithi177.front.sepia.ceph.com 2023-12-13T20:48:54.685 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-13T20:48:54.702 INFO:teuthology.lock.ops:Updating [smithi006.front.sepia.ceph.com]: set os type and version on server 2023-12-13T20:48:54.702 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-12-13T20:48:54.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-13T20:48:54.766 DEBUG:teuthology.orchestra.run.smithi006:> true 2023-12-13T20:48:54.827 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-12-13T20:48:54.827 DEBUG:teuthology.orchestra.run.smithi006:> uname -m 2023-12-13T20:48:54.881 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64 2023-12-13T20:48:54.882 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release 2023-12-13T20:48:54.935 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="Red Hat Enterprise Linux" 2023-12-13T20:48:54.935 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="8.4 (Ootpa)" 2023-12-13T20:48:54.935 INFO:teuthology.orchestra.run.smithi006.stdout:ID="rhel" 2023-12-13T20:48:54.935 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="fedora" 2023-12-13T20:48:54.935 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="8.4" 2023-12-13T20:48:54.936 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el8" 2023-12-13T20:48:54.936 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-13T20:48:54.936 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31" 2023-12-13T20:48:54.936 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-13T20:48:54.936 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://www.redhat.com/" 2023-12-13T20:48:54.936 INFO:teuthology.orchestra.run.smithi006.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-13T20:48:54.936 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-13T20:48:54.936 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-12-13T20:48:54.936 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-13T20:48:54.936 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-13T20:48:54.936 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-13T20:48:54.936 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-13T20:48:54.937 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2023-12-13T20:48:54.967 INFO:teuthology.lock.ops:Updating [smithi177.front.sepia.ceph.com]: set os type and version on server 2023-12-13T20:48:54.968 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-12-13T20:48:54.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-13T20:48:55.033 DEBUG:teuthology.orchestra.run.smithi177:> true 2023-12-13T20:48:55.109 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-12-13T20:48:55.110 DEBUG:teuthology.orchestra.run.smithi177:> uname -m 2023-12-13T20:48:55.164 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64 2023-12-13T20:48:55.165 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2023-12-13T20:48:55.219 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="Red Hat Enterprise Linux" 2023-12-13T20:48:55.219 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="8.4 (Ootpa)" 2023-12-13T20:48:55.220 INFO:teuthology.orchestra.run.smithi177.stdout:ID="rhel" 2023-12-13T20:48:55.220 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="fedora" 2023-12-13T20:48:55.220 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="8.4" 2023-12-13T20:48:55.220 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el8" 2023-12-13T20:48:55.220 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-13T20:48:55.220 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2023-12-13T20:48:55.220 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-13T20:48:55.220 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://www.redhat.com/" 2023-12-13T20:48:55.220 INFO:teuthology.orchestra.run.smithi177.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-13T20:48:55.220 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-13T20:48:55.220 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-12-13T20:48:55.220 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-13T20:48:55.220 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-13T20:48:55.221 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-13T20:48:55.221 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-13T20:48:55.221 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2023-12-13T20:48:55.251 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-13T20:48:55.336 INFO:teuthology.dispatcher.supervisor:Running job 7490421 2023-12-13T20:48:55.337 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-12-13_18:37:52-rbd-wip-yuri4-testing-2023-12-13-0748-quincy-distro-default-smithi/7490421 --name yuriw-2023-12-13_18:37:52-rbd-wip-yuri4-testing-2023-12-13-0748-quincy-distro-default-smithi --description rbd/pwl-cache/home/{1-base/install 2-cluster/{fix-2 openstack} 3-supported-random-distro$/{rhel_8} 4-cache-path 5-cache-mode/rwl 6-cache-size/1G 7-workloads/recovery conf/{disable-pool-app}} -- /home/teuthworker/archive/yuriw-2023-12-13_18:37:52-rbd-wip-yuri4-testing-2023-12-13-0748-quincy-distro-default-smithi/7490421/orig.config.yaml 2023-12-13T20:48:55.339 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-12-13_18:37:52-rbd-wip-yuri4-testing-2023-12-13-0748-quincy-distro-default-smithi/7490421 2023-12-13T20:48:55.340 INFO:teuthology.dispatcher.supervisor:Job PID: 4009043 2023-12-13T20:48:55.340 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-13T20:50:55.438 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-13T20:52:55.602 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-13T20:54:55.770 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-13T20:56:55.938 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-13T20:58:56.114 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-13T20:58:56.190 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-13T20:58:56.282 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-13T20:58:56.282 INFO:teuthology.nuke:Checking targets against current locks 2023-12-13T20:58:56.335 DEBUG:teuthology.nuke:shortname: smithi006 2023-12-13T20:58:56.335 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-13T20:58:56.359 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-13_18:37:52-rbd-wip-yuri4-testing-2023-12-13-0748-quincy-distro-default-smithi/7490421', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-13 20:42:00.903460', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDgsOX29rBM7+YficMiXzjSgFj1F7RENnlJ9+LzzSvHWODpjkZ+ZBzagBUasy1VN5Hb9AiKbQCtoSiOEoYD4jNg='} 2023-12-13T20:58:56.362 DEBUG:teuthology.nuke:shortname: smithi177 2023-12-13T20:58:56.363 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-13T20:58:56.389 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-13_18:37:52-rbd-wip-yuri4-testing-2023-12-13-0748-quincy-distro-default-smithi/7490421', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-13 20:42:00.902074', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDic1HjjstpCm9WH2AgPBrXdU8vG4N+fEk7KqQTD0TEx09DI5N9ECvCgR1kOVplSlWka0NzeJ7gRKcncBsfWxIU='} 2023-12-13T20:58:56.423 INFO:teuthology.orchestra.console.smithi006:Power off 2023-12-13T20:58:56.423 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-13T20:58:56.447 DEBUG:teuthology.orchestra.console.smithi006:power off output: Chassis Power Control: Down/Off 2023-12-13T20:58:56.448 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-13T20:58:56.462 INFO:teuthology.orchestra.console.smithi177:Power off 2023-12-13T20:58:56.462 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-13T20:58:56.474 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is off 2023-12-13T20:58:56.485 DEBUG:teuthology.orchestra.console.smithi177:power off output: Chassis Power Control: Down/Off 2023-12-13T20:58:56.485 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-13T20:58:56.506 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off 2023-12-13T20:58:56.575 INFO:teuthology.orchestra.console.smithi006:Power off completed 2023-12-13T20:58:56.608 INFO:teuthology.orchestra.console.smithi177:Power off completed 2023-12-13T20:58:56.722 INFO:teuthology.lock.ops:unlocked: smithi006.front.sepia.ceph.com 2023-12-13T20:58:56.746 INFO:teuthology.lock.ops:unlocked: smithi177.front.sepia.ceph.com