2024-01-03T02:10:37.995 INFO:root:teuthology version: 0.0.1.dev228+g01dd171
2024-01-03T02:10:37.996 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-03T02:10:38.137 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fb509e71280>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fb50adcc940>, signals=[15])
2024-01-03T02:10:38.137 INFO:teuthology.lock.ops:Start node 'smithi062.front.sepia.ceph.com' reimaging
2024-01-03T02:10:38.137 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: reset os type and version on server
2024-01-03T02:10:38.137 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server
2024-01-03T02:10:38.160 INFO:teuthology.lock.ops:Start node 'smithi184.front.sepia.ceph.com' reimaging
2024-01-03T02:10:38.160 INFO:teuthology.lock.ops:Updating [smithi184.front.sepia.ceph.com]: reset os type and version on server
2024-01-03T02:10:38.160 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server
2024-01-03T02:10:38.275 DEBUG:teuthology.provision.fog.smithi062:Requesting image smithi_centos_9.stream (ID 10)
2024-01-03T02:10:38.292 DEBUG:teuthology.provision.fog.smithi184:Requesting image smithi_centos_9.stream (ID 10)
2024-01-03T02:10:38.321 INFO:teuthology.provision.fog.smithi062:Scheduling deploy of centos 9.stream
2024-01-03T02:10:38.332 INFO:teuthology.provision.fog.smithi184:Scheduling deploy of centos 9.stream
2024-01-03T02:10:38.578 INFO:teuthology.orchestra.console.smithi062:Power off
2024-01-03T02:10:38.578 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-03T02:10:38.599 INFO:teuthology.orchestra.console.smithi184:Power off
2024-01-03T02:10:38.599 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-03T02:10:38.609 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off
2024-01-03T02:10:38.610 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-03T02:10:38.622 DEBUG:teuthology.orchestra.console.smithi184:power off output: Chassis Power Control: Down/Off
2024-01-03T02:10:38.623 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-03T02:10:38.635 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on
2024-01-03T02:10:38.648 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on
2024-01-03T02:10:42.640 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-03T02:10:42.650 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-03T02:10:42.749 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on
2024-01-03T02:10:42.758 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on
2024-01-03T02:10:46.752 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-03T02:10:46.760 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-03T02:10:46.861 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on
2024-01-03T02:10:46.868 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is off
2024-01-03T02:10:46.968 INFO:teuthology.orchestra.console.smithi184:Power off completed
2024-01-03T02:10:47.069 INFO:teuthology.orchestra.console.smithi184:Power on
2024-01-03T02:10:47.069 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-03T02:10:47.090 DEBUG:teuthology.orchestra.console.smithi184:power on output: Chassis Power Control: Up/On
2024-01-03T02:10:47.090 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-03T02:10:47.112 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is off
2024-01-03T02:10:50.864 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-03T02:10:50.973 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off
2024-01-03T02:10:51.073 INFO:teuthology.orchestra.console.smithi062:Power off completed
2024-01-03T02:10:51.113 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-03T02:10:51.174 INFO:teuthology.orchestra.console.smithi062:Power on
2024-01-03T02:10:51.175 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-03T02:10:51.197 DEBUG:teuthology.orchestra.console.smithi062:power on output: Chassis Power Control: Up/On
2024-01-03T02:10:51.198 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-03T02:10:51.220 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is off
2024-01-03T02:10:51.221 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off
2024-01-03T02:10:55.224 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-03T02:10:55.232 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-03T02:10:55.334 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on
2024-01-03T02:10:55.340 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on
2024-01-03T02:10:55.436 INFO:teuthology.orchestra.console.smithi184:Power on completed
2024-01-03T02:10:55.441 INFO:teuthology.orchestra.console.smithi062:Power on completed
2024-01-03T02:10:55.537 INFO:teuthology.provision.fog.smithi184:Waiting for deploy to finish
2024-01-03T02:10:55.542 INFO:teuthology.provision.fog.smithi062:Waiting for deploy to finish
2024-01-03T02:13:26.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-03T02:13:41.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-03T02:14:00.256 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62
2024-01-03T02:14:06.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-03T02:14:09.308 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62
2024-01-03T02:14:15.309 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-03T02:14:18.367 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62
2024-01-03T02:14:24.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-03T02:14:27.424 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62
2024-01-03T02:14:33.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-03T02:14:57.181 ERROR:teuthology.orchestra.connection:Error authenticating with smithi184.front.sepia.ceph.com: AuthenticationException('Authentication timeout.')
2024-01-03T02:15:16.607 ERROR:teuthology.orchestra.connection:Error authenticating with smithi184.front.sepia.ceph.com: BadHostKeyException('smithi184.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fb5098fa220>, <paramiko.ed25519key.Ed25519Key object at 0x7fb509a54250>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi184.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO0iPPOpbDQ+zqJl8Hf02kn7a0t2KQsUwfLbqcFL/MMyM9nNbKIgkCzG/rirsUR0EIM+PrZgCZ8INQRe3wNeYCY=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIODEywkXgeEAIsprFFQVCYgRMSMKAKRy2vqEWWoX/zrh'
2024-01-03T02:15:23.626 ERROR:teuthology.orchestra.connection:Error authenticating with smithi184.front.sepia.ceph.com: BadHostKeyException('smithi184.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fb5098fad00>, <paramiko.ed25519key.Ed25519Key object at 0x7fb509a54250>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi184.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO0iPPOpbDQ+zqJl8Hf02kn7a0t2KQsUwfLbqcFL/MMyM9nNbKIgkCzG/rirsUR0EIM+PrZgCZ8INQRe3wNeYCY=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIODEywkXgeEAIsprFFQVCYgRMSMKAKRy2vqEWWoX/zrh'
2024-01-03T02:15:33.429 WARNING:teuthology.provision.fog.smithi062:timed out
2024-01-03T02:15:33.646 ERROR:teuthology.orchestra.connection:Error authenticating with smithi184.front.sepia.ceph.com: BadHostKeyException('smithi184.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fb509a6a0a0>, <paramiko.ed25519key.Ed25519Key object at 0x7fb509a54250>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi184.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO0iPPOpbDQ+zqJl8Hf02kn7a0t2KQsUwfLbqcFL/MMyM9nNbKIgkCzG/rirsUR0EIM+PrZgCZ8INQRe3wNeYCY=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIODEywkXgeEAIsprFFQVCYgRMSMKAKRy2vqEWWoX/zrh'
2024-01-03T02:15:39.431 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-03T02:15:39.712 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-03T02:15:39.713 DEBUG:teuthology.orchestra.run.smithi062:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-03T02:15:40.056 INFO:teuthology.provision.fog.smithi062:Node is ready
2024-01-03T02:15:40.057 DEBUG:teuthology.orchestra.run.smithi062:> hostname
2024-01-03T02:15:40.072 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062.front.sepia.ceph.com
2024-01-03T02:15:40.072 DEBUG:teuthology.orchestra.run.smithi062:> grep smithi062.front.sepia.ceph.com /etc/hosts
2024-01-03T02:15:40.129 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62 smithi062.front.sepia.ceph.com smithi062
2024-01-03T02:15:40.129 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname smithi062
2024-01-03T02:15:40.244 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/smithi062.front.sepia.ceph.com/smithi062/g' /etc/hosts
2024-01-03T02:15:40.277 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/172.21.15.62/172.21.15.62/g' /etc/hosts
2024-01-03T02:15:40.349 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/smithi062.front.sepia.ceph.com/smithi062/g' /etc/hostname
2024-01-03T02:15:40.421 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname smithi062
2024-01-03T02:15:40.492 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release
2024-01-03T02:15:40.547 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream"
2024-01-03T02:15:40.547 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9"
2024-01-03T02:15:40.547 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos"
2024-01-03T02:15:40.548 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora"
2024-01-03T02:15:40.548 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9"
2024-01-03T02:15:40.548 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9"
2024-01-03T02:15:40.548 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-03T02:15:40.548 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31"
2024-01-03T02:15:40.548 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon"
2024-01-03T02:15:40.548 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-03T02:15:40.548 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/"
2024-01-03T02:15:40.548 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-03T02:15:40.548 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-03T02:15:40.548 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-03T02:15:40.549 INFO:teuthology.provision.fog.smithi062:Deploy complete!
2024-01-03T02:15:40.550 DEBUG:teuthology.parallel:result is None
2024-01-03T02:15:46.665 ERROR:teuthology.orchestra.connection:Error authenticating with smithi184.front.sepia.ceph.com: BadHostKeyException('smithi184.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fb5098c21c0>, <paramiko.ed25519key.Ed25519Key object at 0x7fb509a54250>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi184.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO0iPPOpbDQ+zqJl8Hf02kn7a0t2KQsUwfLbqcFL/MMyM9nNbKIgkCzG/rirsUR0EIM+PrZgCZ8INQRe3wNeYCY=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIODEywkXgeEAIsprFFQVCYgRMSMKAKRy2vqEWWoX/zrh'
2024-01-03T02:16:02.684 ERROR:teuthology.orchestra.connection:Error authenticating with smithi184.front.sepia.ceph.com: BadHostKeyException('smithi184.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fb509a6a580>, <paramiko.ed25519key.Ed25519Key object at 0x7fb509a54250>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi184.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO0iPPOpbDQ+zqJl8Hf02kn7a0t2KQsUwfLbqcFL/MMyM9nNbKIgkCzG/rirsUR0EIM+PrZgCZ8INQRe3wNeYCY=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIODEywkXgeEAIsprFFQVCYgRMSMKAKRy2vqEWWoX/zrh'
2024-01-03T02:16:21.703 ERROR:teuthology.orchestra.connection:Error authenticating with smithi184.front.sepia.ceph.com: BadHostKeyException('smithi184.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fb5098c27c0>, <paramiko.ed25519key.Ed25519Key object at 0x7fb509a54250>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi184.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO0iPPOpbDQ+zqJl8Hf02kn7a0t2KQsUwfLbqcFL/MMyM9nNbKIgkCzG/rirsUR0EIM+PrZgCZ8INQRe3wNeYCY=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIODEywkXgeEAIsprFFQVCYgRMSMKAKRy2vqEWWoX/zrh'
2024-01-03T02:16:43.723 ERROR:teuthology.orchestra.connection:Error authenticating with smithi184.front.sepia.ceph.com: BadHostKeyException('smithi184.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fb5098c4340>, <paramiko.ed25519key.Ed25519Key object at 0x7fb509a54250>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi184.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO0iPPOpbDQ+zqJl8Hf02kn7a0t2KQsUwfLbqcFL/MMyM9nNbKIgkCzG/rirsUR0EIM+PrZgCZ8INQRe3wNeYCY=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIODEywkXgeEAIsprFFQVCYgRMSMKAKRy2vqEWWoX/zrh'
2024-01-03T02:16:44.860 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2024-01-03T02:17:08.741 ERROR:teuthology.orchestra.connection:Error authenticating with smithi184.front.sepia.ceph.com: BadHostKeyException('smithi184.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fb5098c4d60>, <paramiko.ed25519key.Ed25519Key object at 0x7fb509a54250>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi184.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO0iPPOpbDQ+zqJl8Hf02kn7a0t2KQsUwfLbqcFL/MMyM9nNbKIgkCzG/rirsUR0EIM+PrZgCZ8INQRe3wNeYCY=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIODEywkXgeEAIsprFFQVCYgRMSMKAKRy2vqEWWoX/zrh'
2024-01-03T02:17:36.760 ERROR:teuthology.orchestra.connection:Error authenticating with smithi184.front.sepia.ceph.com: BadHostKeyException('smithi184.front.sepia.ceph.com', <paramiko.ecdsakey.ECDSAKey object at 0x7fb5098c9760>, <paramiko.ed25519key.Ed25519Key object at 0x7fb509a54250>) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 438, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi184.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO0iPPOpbDQ+zqJl8Hf02kn7a0t2KQsUwfLbqcFL/MMyM9nNbKIgkCzG/rirsUR0EIM+PrZgCZ8INQRe3wNeYCY=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIODEywkXgeEAIsprFFQVCYgRMSMKAKRy2vqEWWoX/zrh'
2024-01-03T02:17:36.761 WARNING:teuthology.provision.fog.smithi184:'connect to smithi184.front.sepia.ceph.com' reached maximum tries (11) after waiting for 145 seconds
2024-01-03T02:17:42.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-03T02:17:43.032 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-03T02:17:43.032 DEBUG:teuthology.orchestra.run.smithi184:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-03T02:17:43.384 INFO:teuthology.provision.fog.smithi184:Node is ready
2024-01-03T02:17:43.387 DEBUG:teuthology.orchestra.run.smithi184:> hostname
2024-01-03T02:17:43.402 INFO:teuthology.orchestra.run.smithi184.stdout:smithi184.front.sepia.ceph.com
2024-01-03T02:17:43.402 DEBUG:teuthology.orchestra.run.smithi184:> grep smithi184.front.sepia.ceph.com /etc/hosts
2024-01-03T02:17:43.456 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.15.184 smithi184.front.sepia.ceph.com smithi184
2024-01-03T02:17:43.457 DEBUG:teuthology.orchestra.run.smithi184:> sudo hostname smithi184
2024-01-03T02:17:43.564 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i -e 's/smithi184.front.sepia.ceph.com/smithi184/g' /etc/hosts
2024-01-03T02:17:43.592 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i -e 's/172.21.15.184/172.21.15.184/g' /etc/hosts
2024-01-03T02:17:43.660 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i -e 's/smithi184.front.sepia.ceph.com/smithi184/g' /etc/hostname
2024-01-03T02:17:43.727 DEBUG:teuthology.orchestra.run.smithi184:> sudo hostname smithi184
2024-01-03T02:17:43.793 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release
2024-01-03T02:17:43.848 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream"
2024-01-03T02:17:43.848 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9"
2024-01-03T02:17:43.848 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos"
2024-01-03T02:17:43.848 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora"
2024-01-03T02:17:43.848 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9"
2024-01-03T02:17:43.848 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9"
2024-01-03T02:17:43.848 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-03T02:17:43.848 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31"
2024-01-03T02:17:43.848 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon"
2024-01-03T02:17:43.848 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-03T02:17:43.849 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/"
2024-01-03T02:17:43.849 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-03T02:17:43.849 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-03T02:17:43.849 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-03T02:17:43.849 INFO:teuthology.provision.fog.smithi184:Deploy complete!
2024-01-03T02:17:43.850 DEBUG:teuthology.parallel:result is None
2024-01-03T02:17:44.027 INFO:teuthology.lock.ops:Checking smithi062.front.sepia.ceph.com
2024-01-03T02:17:44.027 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-03T02:17:44.050 INFO:teuthology.lock.ops:Checking smithi184.front.sepia.ceph.com
2024-01-03T02:17:44.050 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-03T02:17:44.067 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: set os type and version on server
2024-01-03T02:17:44.067 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com'
2024-01-03T02:17:44.068 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-03T02:17:44.165 DEBUG:teuthology.orchestra.run.smithi062:> true
2024-01-03T02:17:44.346 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com'
2024-01-03T02:17:44.347 DEBUG:teuthology.orchestra.run.smithi062:> uname -m
2024-01-03T02:17:44.402 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64
2024-01-03T02:17:44.402 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release
2024-01-03T02:17:44.457 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream"
2024-01-03T02:17:44.457 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9"
2024-01-03T02:17:44.457 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos"
2024-01-03T02:17:44.457 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora"
2024-01-03T02:17:44.457 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9"
2024-01-03T02:17:44.457 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9"
2024-01-03T02:17:44.457 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-03T02:17:44.457 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31"
2024-01-03T02:17:44.457 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon"
2024-01-03T02:17:44.457 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-03T02:17:44.458 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/"
2024-01-03T02:17:44.458 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-03T02:17:44.458 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-03T02:17:44.458 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-03T02:17:44.458 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server
2024-01-03T02:17:44.483 INFO:teuthology.lock.ops:Updating [smithi184.front.sepia.ceph.com]: set os type and version on server
2024-01-03T02:17:44.483 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com'
2024-01-03T02:17:44.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-03T02:17:44.566 DEBUG:teuthology.orchestra.run.smithi184:> true
2024-01-03T02:17:44.630 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com'
2024-01-03T02:17:44.631 DEBUG:teuthology.orchestra.run.smithi184:> uname -m
2024-01-03T02:17:44.685 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64
2024-01-03T02:17:44.685 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release
2024-01-03T02:17:44.740 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream"
2024-01-03T02:17:44.740 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9"
2024-01-03T02:17:44.740 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos"
2024-01-03T02:17:44.740 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora"
2024-01-03T02:17:44.740 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9"
2024-01-03T02:17:44.741 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9"
2024-01-03T02:17:44.741 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-03T02:17:44.741 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31"
2024-01-03T02:17:44.741 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon"
2024-01-03T02:17:44.741 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-03T02:17:44.741 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/"
2024-01-03T02:17:44.741 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-03T02:17:44.741 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-03T02:17:44.741 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-03T02:17:44.742 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server
2024-01-03T02:17:44.766 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-03T02:17:44.855 INFO:teuthology.dispatcher.supervisor:Running job 7504637
2024-01-03T02:17:44.855 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/virtualenv/bin/teuthology -v --owner scheduled_sjust@teuthology --archive /home/teuthworker/archive/sjust-2024-01-03_01:38:51-rados:thrash-erasure-code-wip-sjust-msr-testing-2024-01-02-distro-default-smithi/7504637 --name sjust-2024-01-03_01:38:51-rados:thrash-erasure-code-wip-sjust-msr-testing-2024-01-02-distro-default-smithi --description rados:thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-balanced} -- /home/teuthworker/archive/sjust-2024-01-03_01:38:51-rados:thrash-erasure-code-wip-sjust-msr-testing-2024-01-02-distro-default-smithi/7504637/orig.config.yaml
2024-01-03T02:17:44.859 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sjust-2024-01-03_01:38:51-rados:thrash-erasure-code-wip-sjust-msr-testing-2024-01-02-distro-default-smithi/7504637
2024-01-03T02:17:44.859 INFO:teuthology.dispatcher.supervisor:Job PID: 3206640
2024-01-03T02:17:44.859 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-01-03T02:19:44.888 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-03T02:21:45.052 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-03T02:23:45.217 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-03T02:25:45.381 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-03T02:27:45.544 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-03T02:29:45.740 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-03T02:31:45.913 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-03T02:33:46.076 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-03T02:35:46.242 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-03T02:37:46.405 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-03T02:37:46.475 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2024-01-03T02:37:46.571 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2024-01-03T02:37:46.572 INFO:teuthology.nuke:Checking targets against current locks
2024-01-03T02:37:46.614 DEBUG:teuthology.nuke:shortname: smithi062
2024-01-03T02:37:46.615 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-03T02:37:46.636 DEBUG:teuthology.nuke:shortname: smithi184
2024-01-03T02:37:46.637 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-03T02:37:46.641 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sjust-2024-01-03_01:38:51-rados:thrash-erasure-code-wip-sjust-msr-testing-2024-01-02-distro-default-smithi/7504637', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-03 02:10:36.965686', 'locked_by': 'scheduled_sjust@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPq55ULp5lANoHkImBIf0TPdWyiowzXhn29TNiSmSRnDyQQ2EUSloRyiFttoOAHpb3Ug5bVr5ncZq59OSKxovkc='}
2024-01-03T02:37:46.663 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sjust-2024-01-03_01:38:51-rados:thrash-erasure-code-wip-sjust-msr-testing-2024-01-02-distro-default-smithi/7504637', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-03 02:10:36.967235', 'locked_by': 'scheduled_sjust@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO0iPPOpbDQ+zqJl8Hf02kn7a0t2KQsUwfLbqcFL/MMyM9nNbKIgkCzG/rirsUR0EIM+PrZgCZ8INQRe3wNeYCY='}
2024-01-03T02:37:46.697 INFO:teuthology.orchestra.console.smithi062:Power off
2024-01-03T02:37:46.697 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-03T02:37:46.720 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off
2024-01-03T02:37:46.720 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-03T02:37:46.736 INFO:teuthology.orchestra.console.smithi184:Power off
2024-01-03T02:37:46.736 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-03T02:37:46.745 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off
2024-01-03T02:37:46.757 DEBUG:teuthology.orchestra.console.smithi184:power off output: Chassis Power Control: Down/Off
2024-01-03T02:37:46.757 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-03T02:37:46.779 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is off
2024-01-03T02:37:46.846 INFO:teuthology.orchestra.console.smithi062:Power off completed
2024-01-03T02:37:46.880 INFO:teuthology.orchestra.console.smithi184:Power off completed
2024-01-03T02:37:46.997 INFO:teuthology.lock.ops:unlocked: smithi062.front.sepia.ceph.com
2024-01-03T02:37:47.030 INFO:teuthology.lock.ops:unlocked: smithi184.front.sepia.ceph.com