2024-02-16T15:27:01.547 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55
2024-02-16T15:27:01.548 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-16T15:27:01.693 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f4527033fd0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f4527ed5c10>, signals=[15])
2024-02-16T15:27:01.693 INFO:teuthology.lock.ops:Start node 'smithi115.front.sepia.ceph.com' reimaging
2024-02-16T15:27:01.694 INFO:teuthology.lock.ops:Updating [smithi115.front.sepia.ceph.com]: reset os type and version on server
2024-02-16T15:27:01.694 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server
2024-02-16T15:27:01.716 INFO:teuthology.lock.ops:Start node 'smithi195.front.sepia.ceph.com' reimaging
2024-02-16T15:27:01.716 INFO:teuthology.lock.ops:Updating [smithi195.front.sepia.ceph.com]: reset os type and version on server
2024-02-16T15:27:01.716 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server
2024-02-16T15:27:01.837 DEBUG:teuthology.provision.fog.smithi115:Requesting image smithi_centos_9.stream (ID 10)
2024-02-16T15:27:01.857 DEBUG:teuthology.provision.fog.smithi195:Requesting image smithi_centos_9.stream (ID 10)
2024-02-16T15:27:01.888 INFO:teuthology.provision.fog.smithi115:Scheduling deploy of centos 9.stream
2024-02-16T15:27:01.902 INFO:teuthology.provision.fog.smithi195:Scheduling deploy of centos 9.stream
2024-02-16T15:27:02.126 INFO:teuthology.orchestra.console.smithi115:Power off
2024-02-16T15:27:02.127 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-02-16T15:27:02.140 INFO:teuthology.orchestra.console.smithi195:Power off
2024-02-16T15:27:02.140 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-02-16T15:27:02.149 DEBUG:teuthology.orchestra.console.smithi115:power off output: Chassis Power Control: Down/Off
2024-02-16T15:27:02.149 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-16T15:27:02.161 DEBUG:teuthology.orchestra.console.smithi195:power off output: Chassis Power Control: Down/Off
2024-02-16T15:27:02.161 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-16T15:27:02.169 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is off
2024-02-16T15:27:02.181 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on
2024-02-16T15:27:02.271 INFO:teuthology.orchestra.console.smithi115:Power off completed
2024-02-16T15:27:02.372 INFO:teuthology.orchestra.console.smithi115:Power on
2024-02-16T15:27:02.372 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-02-16T15:27:02.395 DEBUG:teuthology.orchestra.console.smithi115:power on output: Chassis Power Control: Up/On
2024-02-16T15:27:02.396 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-16T15:27:02.415 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is off
2024-02-16T15:27:06.186 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-16T15:27:06.294 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on
2024-02-16T15:27:06.417 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-16T15:27:06.526 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is off
2024-02-16T15:27:10.296 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-16T15:27:10.405 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off
2024-02-16T15:27:10.506 INFO:teuthology.orchestra.console.smithi195:Power off completed
2024-02-16T15:27:10.527 DEBUG:teuthology.orchestra.console.smithi115:pexpect command: ipmitool -H smithi115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-16T15:27:10.606 INFO:teuthology.orchestra.console.smithi195:Power on
2024-02-16T15:27:10.607 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-02-16T15:27:10.629 DEBUG:teuthology.orchestra.console.smithi195:power on output: Chassis Power Control: Up/On
2024-02-16T15:27:10.629 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-16T15:27:10.637 DEBUG:teuthology.orchestra.console.smithi115:check power output: Chassis Power is on
2024-02-16T15:27:10.652 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off
2024-02-16T15:27:10.739 INFO:teuthology.orchestra.console.smithi115:Power on completed
2024-02-16T15:27:10.840 INFO:teuthology.provision.fog.smithi115:Waiting for deploy to finish
2024-02-16T15:27:14.656 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-16T15:27:14.765 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off
2024-02-16T15:27:18.770 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-16T15:27:18.879 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on
2024-02-16T15:27:18.980 INFO:teuthology.orchestra.console.smithi195:Power on completed
2024-02-16T15:27:19.080 INFO:teuthology.provision.fog.smithi195:Waiting for deploy to finish
2024-02-16T15:29:41.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-16T15:29:49.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-16T15:29:52.828 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195
2024-02-16T15:29:58.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-16T15:30:01.884 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195
2024-02-16T15:30:07.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-16T15:30:10.940 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195
2024-02-16T15:30:16.942 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-16T15:30:19.996 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195
2024-02-16T15:30:25.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-16T15:30:29.052 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195
2024-02-16T15:30:35.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-16T15:30:38.108 WARNING:teuthology.provision.fog.smithi195:[Errno None] Unable to connect to port 22 on 172.21.15.195
2024-02-16T15:30:44.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-16T15:31:12.176 ERROR:teuthology.orchestra.connection:Error authenticating with smithi115.front.sepia.ceph.com: AuthenticationException('Authentication timeout.')
2024-02-16T15:31:31.559 ERROR:teuthology.orchestra.connection:Error authenticating with smithi115.front.sepia.ceph.com: BadHostKeyException('smithi115.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:uuQnMw9I1VPmU6JQ+3UOmAcIeG7qKuW1ryVAbrppUa8), PKey(alg=ED25519, bits=256, fp=SHA256:dk49IGGKqXCoN9IeK3Sw40m8UBwD5DVv6TmJ326QOss)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi115.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJbOoCjzdGccAaFZ+6r2mQat4LDJ/lgToErqVbUzCzot7o7q+Beg5fv7XBEOAMOvgU04OM6q03COxpBKnZox4hk=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIBARDOj2i4YHoV9rGuJG2kmI+d91ELQFOi4pNgkRzC5p'
2024-02-16T15:31:38.580 ERROR:teuthology.orchestra.connection:Error authenticating with smithi115.front.sepia.ceph.com: BadHostKeyException('smithi115.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:uuQnMw9I1VPmU6JQ+3UOmAcIeG7qKuW1ryVAbrppUa8), PKey(alg=ED25519, bits=256, fp=SHA256:dk49IGGKqXCoN9IeK3Sw40m8UBwD5DVv6TmJ326QOss)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi115.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJbOoCjzdGccAaFZ+6r2mQat4LDJ/lgToErqVbUzCzot7o7q+Beg5fv7XBEOAMOvgU04OM6q03COxpBKnZox4hk=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIBARDOj2i4YHoV9rGuJG2kmI+d91ELQFOi4pNgkRzC5p'
2024-02-16T15:31:40.732 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2024-02-16T15:31:44.115 WARNING:teuthology.provision.fog.smithi195:timed out
2024-02-16T15:31:48.599 ERROR:teuthology.orchestra.connection:Error authenticating with smithi115.front.sepia.ceph.com: BadHostKeyException('smithi115.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:uuQnMw9I1VPmU6JQ+3UOmAcIeG7qKuW1ryVAbrppUa8), PKey(alg=ED25519, bits=256, fp=SHA256:dk49IGGKqXCoN9IeK3Sw40m8UBwD5DVv6TmJ326QOss)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi115.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJbOoCjzdGccAaFZ+6r2mQat4LDJ/lgToErqVbUzCzot7o7q+Beg5fv7XBEOAMOvgU04OM6q03COxpBKnZox4hk=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIBARDOj2i4YHoV9rGuJG2kmI+d91ELQFOi4pNgkRzC5p'
2024-02-16T15:31:50.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-16T15:31:50.370 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-02-16T15:31:50.371 DEBUG:teuthology.orchestra.run.smithi195:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-02-16T15:31:50.701 INFO:teuthology.provision.fog.smithi195:Node is ready
2024-02-16T15:31:50.702 DEBUG:teuthology.orchestra.run.smithi195:> hostname
2024-02-16T15:31:50.716 INFO:teuthology.orchestra.run.smithi195.stdout:smithi195.front.sepia.ceph.com
2024-02-16T15:31:50.716 DEBUG:teuthology.orchestra.run.smithi195:> grep smithi195.front.sepia.ceph.com /etc/hosts
2024-02-16T15:31:50.772 INFO:teuthology.orchestra.run.smithi195.stdout:172.21.15.195 smithi195.front.sepia.ceph.com smithi195
2024-02-16T15:31:50.773 DEBUG:teuthology.orchestra.run.smithi195:> sudo hostname smithi195
2024-02-16T15:31:50.870 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i -e 's/smithi195.front.sepia.ceph.com/smithi195/g' /etc/hosts
2024-02-16T15:31:50.898 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i -e 's/172.21.15.195/172.21.15.195/g' /etc/hosts
2024-02-16T15:31:50.965 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i -e 's/smithi195.front.sepia.ceph.com/smithi195/g' /etc/hostname
2024-02-16T15:31:51.031 DEBUG:teuthology.orchestra.run.smithi195:> sudo hostname smithi195
2024-02-16T15:31:51.097 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release
2024-02-16T15:31:51.153 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream"
2024-02-16T15:31:51.153 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="9"
2024-02-16T15:31:51.153 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos"
2024-02-16T15:31:51.153 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora"
2024-02-16T15:31:51.153 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="9"
2024-02-16T15:31:51.153 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el9"
2024-02-16T15:31:51.153 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-16T15:31:51.153 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31"
2024-02-16T15:31:51.153 INFO:teuthology.orchestra.run.smithi195.stdout:LOGO="fedora-logo-icon"
2024-02-16T15:31:51.153 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-16T15:31:51.153 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/"
2024-02-16T15:31:51.153 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-16T15:31:51.153 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-16T15:31:51.154 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-16T15:31:51.154 INFO:teuthology.provision.fog.smithi195:Deploy complete!
2024-02-16T15:31:51.156 DEBUG:teuthology.parallel:result is None
2024-02-16T15:32:01.619 ERROR:teuthology.orchestra.connection:Error authenticating with smithi115.front.sepia.ceph.com: BadHostKeyException('smithi115.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:uuQnMw9I1VPmU6JQ+3UOmAcIeG7qKuW1ryVAbrppUa8), PKey(alg=ED25519, bits=256, fp=SHA256:dk49IGGKqXCoN9IeK3Sw40m8UBwD5DVv6TmJ326QOss)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi115.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJbOoCjzdGccAaFZ+6r2mQat4LDJ/lgToErqVbUzCzot7o7q+Beg5fv7XBEOAMOvgU04OM6q03COxpBKnZox4hk=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIBARDOj2i4YHoV9rGuJG2kmI+d91ELQFOi4pNgkRzC5p'
2024-02-16T15:32:17.638 ERROR:teuthology.orchestra.connection:Error authenticating with smithi115.front.sepia.ceph.com: BadHostKeyException('smithi115.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:uuQnMw9I1VPmU6JQ+3UOmAcIeG7qKuW1ryVAbrppUa8), PKey(alg=ED25519, bits=256, fp=SHA256:dk49IGGKqXCoN9IeK3Sw40m8UBwD5DVv6TmJ326QOss)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi115.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJbOoCjzdGccAaFZ+6r2mQat4LDJ/lgToErqVbUzCzot7o7q+Beg5fv7XBEOAMOvgU04OM6q03COxpBKnZox4hk=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIBARDOj2i4YHoV9rGuJG2kmI+d91ELQFOi4pNgkRzC5p'
2024-02-16T15:32:36.658 ERROR:teuthology.orchestra.connection:Error authenticating with smithi115.front.sepia.ceph.com: BadHostKeyException('smithi115.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:uuQnMw9I1VPmU6JQ+3UOmAcIeG7qKuW1ryVAbrppUa8), PKey(alg=ED25519, bits=256, fp=SHA256:dk49IGGKqXCoN9IeK3Sw40m8UBwD5DVv6TmJ326QOss)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi115.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJbOoCjzdGccAaFZ+6r2mQat4LDJ/lgToErqVbUzCzot7o7q+Beg5fv7XBEOAMOvgU04OM6q03COxpBKnZox4hk=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIBARDOj2i4YHoV9rGuJG2kmI+d91ELQFOi4pNgkRzC5p'
2024-02-16T15:32:58.677 ERROR:teuthology.orchestra.connection:Error authenticating with smithi115.front.sepia.ceph.com: BadHostKeyException('smithi115.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:uuQnMw9I1VPmU6JQ+3UOmAcIeG7qKuW1ryVAbrppUa8), PKey(alg=ED25519, bits=256, fp=SHA256:dk49IGGKqXCoN9IeK3Sw40m8UBwD5DVv6TmJ326QOss)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi115.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJbOoCjzdGccAaFZ+6r2mQat4LDJ/lgToErqVbUzCzot7o7q+Beg5fv7XBEOAMOvgU04OM6q03COxpBKnZox4hk=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIBARDOj2i4YHoV9rGuJG2kmI+d91ELQFOi4pNgkRzC5p'
2024-02-16T15:33:23.698 ERROR:teuthology.orchestra.connection:Error authenticating with smithi115.front.sepia.ceph.com: BadHostKeyException('smithi115.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:uuQnMw9I1VPmU6JQ+3UOmAcIeG7qKuW1ryVAbrppUa8), PKey(alg=ED25519, bits=256, fp=SHA256:dk49IGGKqXCoN9IeK3Sw40m8UBwD5DVv6TmJ326QOss)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi115.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJbOoCjzdGccAaFZ+6r2mQat4LDJ/lgToErqVbUzCzot7o7q+Beg5fv7XBEOAMOvgU04OM6q03COxpBKnZox4hk=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIBARDOj2i4YHoV9rGuJG2kmI+d91ELQFOi4pNgkRzC5p'
2024-02-16T15:33:51.717 ERROR:teuthology.orchestra.connection:Error authenticating with smithi115.front.sepia.ceph.com: BadHostKeyException('smithi115.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:uuQnMw9I1VPmU6JQ+3UOmAcIeG7qKuW1ryVAbrppUa8), PKey(alg=ED25519, bits=256, fp=SHA256:dk49IGGKqXCoN9IeK3Sw40m8UBwD5DVv6TmJ326QOss)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi115.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJbOoCjzdGccAaFZ+6r2mQat4LDJ/lgToErqVbUzCzot7o7q+Beg5fv7XBEOAMOvgU04OM6q03COxpBKnZox4hk=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIBARDOj2i4YHoV9rGuJG2kmI+d91ELQFOi4pNgkRzC5p'
2024-02-16T15:33:51.718 WARNING:teuthology.provision.fog.smithi115:'connect to smithi115.front.sepia.ceph.com' reached maximum tries (11) after waiting for 145 seconds
2024-02-16T15:33:57.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-16T15:33:57.894 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-02-16T15:33:57.894 DEBUG:teuthology.orchestra.run.smithi115:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-02-16T15:33:58.219 INFO:teuthology.provision.fog.smithi115:Node is ready
2024-02-16T15:33:58.220 DEBUG:teuthology.orchestra.run.smithi115:> hostname
2024-02-16T15:33:58.234 INFO:teuthology.orchestra.run.smithi115.stdout:smithi115.front.sepia.ceph.com
2024-02-16T15:33:58.235 DEBUG:teuthology.orchestra.run.smithi115:> grep smithi115.front.sepia.ceph.com /etc/hosts
2024-02-16T15:33:58.291 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115 smithi115.front.sepia.ceph.com smithi115
2024-02-16T15:33:58.291 DEBUG:teuthology.orchestra.run.smithi115:> sudo hostname smithi115
2024-02-16T15:33:58.421 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i -e 's/smithi115.front.sepia.ceph.com/smithi115/g' /etc/hosts
2024-02-16T15:33:58.449 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i -e 's/172.21.15.115/172.21.15.115/g' /etc/hosts
2024-02-16T15:33:58.516 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i -e 's/smithi115.front.sepia.ceph.com/smithi115/g' /etc/hostname
2024-02-16T15:33:58.585 DEBUG:teuthology.orchestra.run.smithi115:> sudo hostname smithi115
2024-02-16T15:33:58.652 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release
2024-02-16T15:33:58.706 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Stream"
2024-02-16T15:33:58.707 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="9"
2024-02-16T15:33:58.707 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos"
2024-02-16T15:33:58.707 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora"
2024-02-16T15:33:58.707 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="9"
2024-02-16T15:33:58.707 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el9"
2024-02-16T15:33:58.707 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-16T15:33:58.707 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31"
2024-02-16T15:33:58.707 INFO:teuthology.orchestra.run.smithi115.stdout:LOGO="fedora-logo-icon"
2024-02-16T15:33:58.707 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-16T15:33:58.707 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://centos.org/"
2024-02-16T15:33:58.707 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-16T15:33:58.707 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-16T15:33:58.708 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-16T15:33:58.708 INFO:teuthology.provision.fog.smithi115:Deploy complete!
2024-02-16T15:33:58.709 DEBUG:teuthology.parallel:result is None
2024-02-16T15:33:58.871 INFO:teuthology.lock.ops:Checking smithi115.front.sepia.ceph.com
2024-02-16T15:33:58.871 INFO:teuthology.lock.ops:New key found. Updating...
2024-02-16T15:33:58.902 INFO:teuthology.lock.ops:Checking smithi195.front.sepia.ceph.com
2024-02-16T15:33:58.902 INFO:teuthology.lock.ops:New key found. Updating...
2024-02-16T15:33:58.919 INFO:teuthology.lock.ops:Updating [smithi115.front.sepia.ceph.com]: set os type and version on server
2024-02-16T15:33:58.920 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com'
2024-02-16T15:33:58.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-16T15:33:59.008 DEBUG:teuthology.orchestra.run.smithi115:> true
2024-02-16T15:33:59.073 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com'
2024-02-16T15:33:59.074 DEBUG:teuthology.orchestra.run.smithi115:> uname -m
2024-02-16T15:33:59.127 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64
2024-02-16T15:33:59.128 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release
2024-02-16T15:33:59.181 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Stream"
2024-02-16T15:33:59.181 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="9"
2024-02-16T15:33:59.181 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos"
2024-02-16T15:33:59.181 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora"
2024-02-16T15:33:59.181 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="9"
2024-02-16T15:33:59.181 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el9"
2024-02-16T15:33:59.182 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-16T15:33:59.182 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31"
2024-02-16T15:33:59.182 INFO:teuthology.orchestra.run.smithi115.stdout:LOGO="fedora-logo-icon"
2024-02-16T15:33:59.182 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-16T15:33:59.182 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://centos.org/"
2024-02-16T15:33:59.182 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-16T15:33:59.182 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-16T15:33:59.182 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-16T15:33:59.183 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server
2024-02-16T15:33:59.202 INFO:teuthology.lock.ops:Updating [smithi195.front.sepia.ceph.com]: set os type and version on server
2024-02-16T15:33:59.202 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com'
2024-02-16T15:33:59.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-16T15:33:59.290 DEBUG:teuthology.orchestra.run.smithi195:> true
2024-02-16T15:33:59.458 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com'
2024-02-16T15:33:59.459 DEBUG:teuthology.orchestra.run.smithi195:> uname -m
2024-02-16T15:33:59.513 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64
2024-02-16T15:33:59.513 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release
2024-02-16T15:33:59.568 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream"
2024-02-16T15:33:59.568 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="9"
2024-02-16T15:33:59.568 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos"
2024-02-16T15:33:59.568 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora"
2024-02-16T15:33:59.569 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="9"
2024-02-16T15:33:59.569 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el9"
2024-02-16T15:33:59.569 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-16T15:33:59.569 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31"
2024-02-16T15:33:59.569 INFO:teuthology.orchestra.run.smithi195.stdout:LOGO="fedora-logo-icon"
2024-02-16T15:33:59.569 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-16T15:33:59.569 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/"
2024-02-16T15:33:59.569 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-16T15:33:59.569 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-16T15:33:59.569 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-16T15:33:59.570 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server
2024-02-16T15:33:59.594 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-16T15:33:59.679 INFO:teuthology.dispatcher.supervisor:Running job 7562433
2024-02-16T15:33:59.679 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-02-16_15:10:58-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562433 --name pdonnell-2024-02-16_15:10:58-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi --description fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/pdonnell-2024-02-16_15:10:58-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562433/orig.config.yaml
2024-02-16T15:33:59.683 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2024-02-16_15:10:58-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562433
2024-02-16T15:33:59.683 INFO:teuthology.dispatcher.supervisor:Job PID: 666063
2024-02-16T15:33:59.683 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-02-16T15:35:59.713 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-16T15:37:59.878 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-16T15:40:00.056 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-16T15:42:00.225 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-16T15:44:00.388 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-16T15:46:00.574 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-16T15:48:00.745 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-16T15:50:00.895 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-16T15:52:01.060 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-16T15:54:01.223 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-16T15:56:01.389 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-16T15:58:01.593 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-16T16:00:01.712 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-16T16:00:01.780 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1