2024-02-21T22:58:11.938 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55
2024-02-21T22:58:11.938 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T22:58:12.086 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f9d43313f10>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f9d44374c10>, signals=[15])
2024-02-21T22:58:12.086 INFO:teuthology.lock.ops:Start node 'smithi114.front.sepia.ceph.com' reimaging
2024-02-21T22:58:12.087 INFO:teuthology.lock.ops:Updating [smithi114.front.sepia.ceph.com]: reset os type and version on server
2024-02-21T22:58:12.087 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server
2024-02-21T22:58:12.103 INFO:teuthology.lock.ops:Start node 'smithi174.front.sepia.ceph.com' reimaging
2024-02-21T22:58:12.104 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: reset os type and version on server
2024-02-21T22:58:12.104 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server
2024-02-21T22:58:12.235 DEBUG:teuthology.provision.fog.smithi174:Requesting image smithi_centos_9.stream (ID 10)
2024-02-21T22:58:12.243 DEBUG:teuthology.provision.fog.smithi114:Requesting image smithi_centos_9.stream (ID 10)
2024-02-21T22:58:12.288 INFO:teuthology.provision.fog.smithi114:Scheduling deploy of centos 9.stream
2024-02-21T22:58:12.291 INFO:teuthology.provision.fog.smithi174:Scheduling deploy of centos 9.stream
2024-02-21T22:58:12.601 INFO:teuthology.orchestra.console.smithi174:Power off
2024-02-21T22:58:12.602 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-02-21T22:58:12.615 INFO:teuthology.orchestra.console.smithi114:Power off
2024-02-21T22:58:12.615 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-02-21T22:58:12.625 DEBUG:teuthology.orchestra.console.smithi174:power off output: Chassis Power Control: Down/Off
2024-02-21T22:58:12.625 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T22:58:12.636 DEBUG:teuthology.orchestra.console.smithi114:power off output: Chassis Power Control: Down/Off
2024-02-21T22:58:12.637 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T22:58:12.646 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on
2024-02-21T22:58:12.660 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on
2024-02-21T22:58:16.649 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T22:58:16.660 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T22:58:16.760 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on
2024-02-21T22:58:16.769 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on
2024-02-21T22:58:20.764 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T22:58:20.772 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T22:58:20.874 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off
2024-02-21T22:58:20.880 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on
2024-02-21T22:58:20.975 INFO:teuthology.orchestra.console.smithi174:Power off completed
2024-02-21T22:58:21.076 INFO:teuthology.orchestra.console.smithi174:Power on
2024-02-21T22:58:21.076 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-02-21T22:58:21.098 DEBUG:teuthology.orchestra.console.smithi174:power on output: Chassis Power Control: Up/On
2024-02-21T22:58:21.099 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T22:58:21.119 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off
2024-02-21T22:58:24.885 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T22:58:24.994 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off
2024-02-21T22:58:25.095 INFO:teuthology.orchestra.console.smithi114:Power off completed
2024-02-21T22:58:25.120 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T22:58:25.196 INFO:teuthology.orchestra.console.smithi114:Power on
2024-02-21T22:58:25.196 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-02-21T22:58:25.215 DEBUG:teuthology.orchestra.console.smithi114:power on output: Chassis Power Control: Up/On
2024-02-21T22:58:25.216 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T22:58:25.228 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off
2024-02-21T22:58:25.236 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off
2024-02-21T22:58:29.232 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T22:58:29.239 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-21T22:58:29.348 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on
2024-02-21T22:58:29.442 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on
2024-02-21T22:58:29.449 INFO:teuthology.orchestra.console.smithi114:Power on completed
2024-02-21T22:58:29.543 INFO:teuthology.orchestra.console.smithi174:Power on completed
2024-02-21T22:58:29.550 INFO:teuthology.provision.fog.smithi114:Waiting for deploy to finish
2024-02-21T22:58:29.644 INFO:teuthology.provision.fog.smithi174:Waiting for deploy to finish
2024-02-21T23:01:00.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T23:01:00.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T23:01:03.900 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174
2024-02-21T23:01:09.901 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T23:01:12.956 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174
2024-02-21T23:01:18.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T23:01:22.012 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174
2024-02-21T23:01:28.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T23:01:31.068 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174
2024-02-21T23:01:37.070 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T23:01:40.124 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174
2024-02-21T23:01:46.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T23:01:49.180 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174
2024-02-21T23:01:55.181 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T23:01:58.236 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174
2024-02-21T23:02:04.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T23:02:31.179 ERROR:teuthology.orchestra.connection:Error authenticating with smithi114.front.sepia.ceph.com: AuthenticationException('Authentication timeout.')
2024-02-21T23:02:50.479 ERROR:teuthology.orchestra.connection:Error authenticating with smithi114.front.sepia.ceph.com: BadHostKeyException('smithi114.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:r3L6ZnKPKzjOnp2MUV+2nt8x9lA5aMhghnDsmuC+4bc), PKey(alg=ED25519, bits=256, fp=SHA256:N4vPMivU9HnOeeqCDONWh+UkzQGddO83TFGk+04FUVk)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi114.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMFycTRgH9VlU3ItVDUdHPh1KEFQXy3eX0hty7ituFctDMJNa5wFjZHIh3G6GOp9qI2CCiWGQfZsKJwz1yE27d4=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIM8x+eKkvBzckQ0aG8cG43T4Yll4JtOVt/6eIPfbxaz7'
2024-02-21T23:02:52.220 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2024-02-21T23:02:57.656 ERROR:teuthology.orchestra.connection:Error authenticating with smithi114.front.sepia.ceph.com: BadHostKeyException('smithi114.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:r3L6ZnKPKzjOnp2MUV+2nt8x9lA5aMhghnDsmuC+4bc), PKey(alg=ED25519, bits=256, fp=SHA256:N4vPMivU9HnOeeqCDONWh+UkzQGddO83TFGk+04FUVk)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi114.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMFycTRgH9VlU3ItVDUdHPh1KEFQXy3eX0hty7ituFctDMJNa5wFjZHIh3G6GOp9qI2CCiWGQfZsKJwz1yE27d4=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIM8x+eKkvBzckQ0aG8cG43T4Yll4JtOVt/6eIPfbxaz7'
2024-02-21T23:03:04.241 WARNING:teuthology.provision.fog.smithi174:timed out
2024-02-21T23:03:07.677 ERROR:teuthology.orchestra.connection:Error authenticating with smithi114.front.sepia.ceph.com: BadHostKeyException('smithi114.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:r3L6ZnKPKzjOnp2MUV+2nt8x9lA5aMhghnDsmuC+4bc), PKey(alg=ED25519, bits=256, fp=SHA256:N4vPMivU9HnOeeqCDONWh+UkzQGddO83TFGk+04FUVk)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi114.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMFycTRgH9VlU3ItVDUdHPh1KEFQXy3eX0hty7ituFctDMJNa5wFjZHIh3G6GOp9qI2CCiWGQfZsKJwz1yE27d4=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIM8x+eKkvBzckQ0aG8cG43T4Yll4JtOVt/6eIPfbxaz7'
2024-02-21T23:03:10.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T23:03:10.492 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-02-21T23:03:10.492 DEBUG:teuthology.orchestra.run.smithi174:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-02-21T23:03:10.814 INFO:teuthology.provision.fog.smithi174:Node is ready
2024-02-21T23:03:10.814 DEBUG:teuthology.orchestra.run.smithi174:> hostname
2024-02-21T23:03:10.870 INFO:teuthology.orchestra.run.smithi174.stdout:smithi174.front.sepia.ceph.com
2024-02-21T23:03:10.870 DEBUG:teuthology.orchestra.run.smithi174:> grep smithi174.front.sepia.ceph.com /etc/hosts
2024-02-21T23:03:10.926 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.15.174 smithi174.front.sepia.ceph.com smithi174
2024-02-21T23:03:10.926 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname smithi174
2024-02-21T23:03:11.055 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/smithi174.front.sepia.ceph.com/smithi174/g' /etc/hosts
2024-02-21T23:03:11.082 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/172.21.15.174/172.21.15.174/g' /etc/hosts
2024-02-21T23:03:11.150 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/smithi174.front.sepia.ceph.com/smithi174/g' /etc/hostname
2024-02-21T23:03:11.218 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname smithi174
2024-02-21T23:03:11.286 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release
2024-02-21T23:03:11.342 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream"
2024-02-21T23:03:11.342 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9"
2024-02-21T23:03:11.342 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos"
2024-02-21T23:03:11.343 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora"
2024-02-21T23:03:11.343 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9"
2024-02-21T23:03:11.343 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9"
2024-02-21T23:03:11.343 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-21T23:03:11.343 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31"
2024-02-21T23:03:11.343 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon"
2024-02-21T23:03:11.343 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-21T23:03:11.343 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/"
2024-02-21T23:03:11.343 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-21T23:03:11.343 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-21T23:03:11.343 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-21T23:03:11.344 INFO:teuthology.provision.fog.smithi174:Deploy complete!
2024-02-21T23:03:11.346 DEBUG:teuthology.parallel:result is None
2024-02-21T23:03:20.696 ERROR:teuthology.orchestra.connection:Error authenticating with smithi114.front.sepia.ceph.com: BadHostKeyException('smithi114.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:r3L6ZnKPKzjOnp2MUV+2nt8x9lA5aMhghnDsmuC+4bc), PKey(alg=ED25519, bits=256, fp=SHA256:N4vPMivU9HnOeeqCDONWh+UkzQGddO83TFGk+04FUVk)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi114.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMFycTRgH9VlU3ItVDUdHPh1KEFQXy3eX0hty7ituFctDMJNa5wFjZHIh3G6GOp9qI2CCiWGQfZsKJwz1yE27d4=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIM8x+eKkvBzckQ0aG8cG43T4Yll4JtOVt/6eIPfbxaz7'
2024-02-21T23:03:36.716 ERROR:teuthology.orchestra.connection:Error authenticating with smithi114.front.sepia.ceph.com: BadHostKeyException('smithi114.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:r3L6ZnKPKzjOnp2MUV+2nt8x9lA5aMhghnDsmuC+4bc), PKey(alg=ED25519, bits=256, fp=SHA256:N4vPMivU9HnOeeqCDONWh+UkzQGddO83TFGk+04FUVk)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi114.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMFycTRgH9VlU3ItVDUdHPh1KEFQXy3eX0hty7ituFctDMJNa5wFjZHIh3G6GOp9qI2CCiWGQfZsKJwz1yE27d4=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIM8x+eKkvBzckQ0aG8cG43T4Yll4JtOVt/6eIPfbxaz7'
2024-02-21T23:03:55.735 ERROR:teuthology.orchestra.connection:Error authenticating with smithi114.front.sepia.ceph.com: BadHostKeyException('smithi114.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:r3L6ZnKPKzjOnp2MUV+2nt8x9lA5aMhghnDsmuC+4bc), PKey(alg=ED25519, bits=256, fp=SHA256:N4vPMivU9HnOeeqCDONWh+UkzQGddO83TFGk+04FUVk)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi114.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMFycTRgH9VlU3ItVDUdHPh1KEFQXy3eX0hty7ituFctDMJNa5wFjZHIh3G6GOp9qI2CCiWGQfZsKJwz1yE27d4=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIM8x+eKkvBzckQ0aG8cG43T4Yll4JtOVt/6eIPfbxaz7'
2024-02-21T23:04:17.754 ERROR:teuthology.orchestra.connection:Error authenticating with smithi114.front.sepia.ceph.com: BadHostKeyException('smithi114.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:r3L6ZnKPKzjOnp2MUV+2nt8x9lA5aMhghnDsmuC+4bc), PKey(alg=ED25519, bits=256, fp=SHA256:N4vPMivU9HnOeeqCDONWh+UkzQGddO83TFGk+04FUVk)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi114.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMFycTRgH9VlU3ItVDUdHPh1KEFQXy3eX0hty7ituFctDMJNa5wFjZHIh3G6GOp9qI2CCiWGQfZsKJwz1yE27d4=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIM8x+eKkvBzckQ0aG8cG43T4Yll4JtOVt/6eIPfbxaz7'
2024-02-21T23:04:42.774 ERROR:teuthology.orchestra.connection:Error authenticating with smithi114.front.sepia.ceph.com: BadHostKeyException('smithi114.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:r3L6ZnKPKzjOnp2MUV+2nt8x9lA5aMhghnDsmuC+4bc), PKey(alg=ED25519, bits=256, fp=SHA256:N4vPMivU9HnOeeqCDONWh+UkzQGddO83TFGk+04FUVk)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi114.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMFycTRgH9VlU3ItVDUdHPh1KEFQXy3eX0hty7ituFctDMJNa5wFjZHIh3G6GOp9qI2CCiWGQfZsKJwz1yE27d4=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIM8x+eKkvBzckQ0aG8cG43T4Yll4JtOVt/6eIPfbxaz7'
2024-02-21T23:05:10.794 ERROR:teuthology.orchestra.connection:Error authenticating with smithi114.front.sepia.ceph.com: BadHostKeyException('smithi114.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:r3L6ZnKPKzjOnp2MUV+2nt8x9lA5aMhghnDsmuC+4bc), PKey(alg=ED25519, bits=256, fp=SHA256:N4vPMivU9HnOeeqCDONWh+UkzQGddO83TFGk+04FUVk)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi114.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMFycTRgH9VlU3ItVDUdHPh1KEFQXy3eX0hty7ituFctDMJNa5wFjZHIh3G6GOp9qI2CCiWGQfZsKJwz1yE27d4=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIM8x+eKkvBzckQ0aG8cG43T4Yll4JtOVt/6eIPfbxaz7'
2024-02-21T23:05:10.795 WARNING:teuthology.provision.fog.smithi114:'connect to smithi114.front.sepia.ceph.com' reached maximum tries (11) after waiting for 145 seconds
2024-02-21T23:05:16.797 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T23:05:17.042 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-02-21T23:05:17.043 DEBUG:teuthology.orchestra.run.smithi114:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-02-21T23:05:17.383 INFO:teuthology.provision.fog.smithi114:Node is ready
2024-02-21T23:05:17.383 DEBUG:teuthology.orchestra.run.smithi114:> hostname
2024-02-21T23:05:17.398 INFO:teuthology.orchestra.run.smithi114.stdout:smithi114.front.sepia.ceph.com
2024-02-21T23:05:17.399 DEBUG:teuthology.orchestra.run.smithi114:> grep smithi114.front.sepia.ceph.com /etc/hosts
2024-02-21T23:05:17.455 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114 smithi114.front.sepia.ceph.com smithi114
2024-02-21T23:05:17.455 DEBUG:teuthology.orchestra.run.smithi114:> sudo hostname smithi114
2024-02-21T23:05:17.558 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i -e 's/smithi114.front.sepia.ceph.com/smithi114/g' /etc/hosts
2024-02-21T23:05:17.587 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i -e 's/172.21.15.114/172.21.15.114/g' /etc/hosts
2024-02-21T23:05:17.655 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i -e 's/smithi114.front.sepia.ceph.com/smithi114/g' /etc/hostname
2024-02-21T23:05:17.724 DEBUG:teuthology.orchestra.run.smithi114:> sudo hostname smithi114
2024-02-21T23:05:17.791 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release
2024-02-21T23:05:17.845 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Stream"
2024-02-21T23:05:17.846 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="9"
2024-02-21T23:05:17.846 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos"
2024-02-21T23:05:17.846 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora"
2024-02-21T23:05:17.846 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="9"
2024-02-21T23:05:17.846 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el9"
2024-02-21T23:05:17.846 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-21T23:05:17.846 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31"
2024-02-21T23:05:17.846 INFO:teuthology.orchestra.run.smithi114.stdout:LOGO="fedora-logo-icon"
2024-02-21T23:05:17.846 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-21T23:05:17.846 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://centos.org/"
2024-02-21T23:05:17.846 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-21T23:05:17.846 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-21T23:05:17.847 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-21T23:05:17.847 INFO:teuthology.provision.fog.smithi114:Deploy complete!
2024-02-21T23:05:17.848 DEBUG:teuthology.parallel:result is None
2024-02-21T23:05:18.011 INFO:teuthology.lock.ops:Checking smithi114.front.sepia.ceph.com
2024-02-21T23:05:18.011 INFO:teuthology.lock.ops:New key found. Updating...
2024-02-21T23:05:18.041 INFO:teuthology.lock.ops:Checking smithi174.front.sepia.ceph.com
2024-02-21T23:05:18.041 INFO:teuthology.lock.ops:New key found. Updating...
2024-02-21T23:05:18.058 INFO:teuthology.lock.ops:Updating [smithi114.front.sepia.ceph.com]: set os type and version on server
2024-02-21T23:05:18.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com'
2024-02-21T23:05:18.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T23:05:18.147 DEBUG:teuthology.orchestra.run.smithi114:> true
2024-02-21T23:05:18.214 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com'
2024-02-21T23:05:18.214 DEBUG:teuthology.orchestra.run.smithi114:> uname -m
2024-02-21T23:05:18.268 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64
2024-02-21T23:05:18.269 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release
2024-02-21T23:05:18.322 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Stream"
2024-02-21T23:05:18.322 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="9"
2024-02-21T23:05:18.322 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos"
2024-02-21T23:05:18.322 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora"
2024-02-21T23:05:18.322 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="9"
2024-02-21T23:05:18.322 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el9"
2024-02-21T23:05:18.323 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-21T23:05:18.323 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31"
2024-02-21T23:05:18.323 INFO:teuthology.orchestra.run.smithi114.stdout:LOGO="fedora-logo-icon"
2024-02-21T23:05:18.323 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-21T23:05:18.323 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://centos.org/"
2024-02-21T23:05:18.323 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-21T23:05:18.323 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-21T23:05:18.323 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-21T23:05:18.324 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server
2024-02-21T23:05:18.357 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: set os type and version on server
2024-02-21T23:05:18.357 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2024-02-21T23:05:18.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-21T23:05:18.447 DEBUG:teuthology.orchestra.run.smithi174:> true
2024-02-21T23:05:18.614 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com'
2024-02-21T23:05:18.614 DEBUG:teuthology.orchestra.run.smithi174:> uname -m
2024-02-21T23:05:18.669 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64
2024-02-21T23:05:18.669 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release
2024-02-21T23:05:18.723 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream"
2024-02-21T23:05:18.723 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9"
2024-02-21T23:05:18.724 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos"
2024-02-21T23:05:18.724 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora"
2024-02-21T23:05:18.724 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9"
2024-02-21T23:05:18.724 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9"
2024-02-21T23:05:18.724 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-21T23:05:18.724 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31"
2024-02-21T23:05:18.724 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon"
2024-02-21T23:05:18.724 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-21T23:05:18.724 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/"
2024-02-21T23:05:18.724 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-21T23:05:18.724 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-21T23:05:18.724 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-21T23:05:18.725 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server
2024-02-21T23:05:18.757 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:05:18.849 INFO:teuthology.dispatcher.supervisor:Running job 7570024
2024-02-21T23:05:18.850 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology -v --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570024 --name leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570024/orig.config.yaml
2024-02-21T23:05:18.853 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/leonidus-2024-02-21_22:31:03-fs-wip-lusov-qdb-distro-default-smithi/7570024
2024-02-21T23:05:18.853 INFO:teuthology.dispatcher.supervisor:Job PID: 2540740
2024-02-21T23:05:18.854 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-02-21T23:07:18.860 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:09:19.031 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:11:19.201 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:13:19.377 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:15:19.553 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:17:19.726 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:19:19.909 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:21:20.068 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:23:20.236 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:25:20.404 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:27:20.577 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:29:20.702 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:31:20.875 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:33:21.044 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:35:21.241 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:37:21.419 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:39:21.558 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:41:21.745 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:43:21.933 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:45:22.101 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:47:22.264 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:49:22.433 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-21T23:49:22.502 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1