2024-04-18T13:20:13.637 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d
2024-04-18T13:20:13.638 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-18T13:20:13.841 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fa3133a2f70>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fa314403c10>, signals=[15])
2024-04-18T13:20:13.841 INFO:teuthology.lock.ops:Start node 'smithi121.front.sepia.ceph.com' reimaging
2024-04-18T13:20:13.842 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: reset os type and version on server
2024-04-18T13:20:13.842 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server
2024-04-18T13:20:13.861 INFO:teuthology.lock.ops:Start node 'smithi122.front.sepia.ceph.com' reimaging
2024-04-18T13:20:13.861 INFO:teuthology.lock.ops:Updating [smithi122.front.sepia.ceph.com]: reset os type and version on server
2024-04-18T13:20:13.861 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server
2024-04-18T13:20:13.887 INFO:teuthology.lock.ops:Start node 'smithi153.front.sepia.ceph.com' reimaging
2024-04-18T13:20:13.887 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: reset os type and version on server
2024-04-18T13:20:13.887 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server
2024-04-18T13:20:13.910 INFO:teuthology.lock.ops:Start node 'smithi203.front.sepia.ceph.com' reimaging
2024-04-18T13:20:13.910 INFO:teuthology.lock.ops:Updating [smithi203.front.sepia.ceph.com]: reset os type and version on server
2024-04-18T13:20:13.910 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server
2024-04-18T13:20:13.971 DEBUG:teuthology.provision.fog.smithi121:Requesting image smithi_centos_9.stream (ID 10)
2024-04-18T13:20:13.993 DEBUG:teuthology.provision.fog.smithi122:Requesting image smithi_centos_9.stream (ID 10)
2024-04-18T13:20:14.007 DEBUG:teuthology.provision.fog.smithi153:Requesting image smithi_centos_9.stream (ID 10)
2024-04-18T13:20:14.010 INFO:teuthology.provision.fog.smithi121:Scheduling deploy of centos 9.stream
2024-04-18T13:20:14.033 INFO:teuthology.provision.fog.smithi122:Scheduling deploy of centos 9.stream
2024-04-18T13:20:14.036 DEBUG:teuthology.provision.fog.smithi203:Requesting image smithi_centos_9.stream (ID 10)
2024-04-18T13:20:14.056 INFO:teuthology.provision.fog.smithi153:Scheduling deploy of centos 9.stream
2024-04-18T13:20:14.076 INFO:teuthology.provision.fog.smithi203:Scheduling deploy of centos 9.stream
2024-04-18T13:20:14.288 INFO:teuthology.orchestra.console.smithi121:Power off
2024-04-18T13:20:14.288 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-18T13:20:14.310 DEBUG:teuthology.orchestra.console.smithi121:power off output: Chassis Power Control: Down/Off
2024-04-18T13:20:14.311 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T13:20:14.331 INFO:teuthology.orchestra.console.smithi153:Power off
2024-04-18T13:20:14.332 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-18T13:20:14.343 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is off
2024-04-18T13:20:14.352 DEBUG:teuthology.orchestra.console.smithi153:power off output: Chassis Power Control: Down/Off
2024-04-18T13:20:14.352 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T13:20:14.370 INFO:teuthology.orchestra.console.smithi122:Power off
2024-04-18T13:20:14.370 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-18T13:20:14.382 INFO:teuthology.orchestra.console.smithi203:Power off
2024-04-18T13:20:14.383 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-18T13:20:14.390 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off
2024-04-18T13:20:14.392 DEBUG:teuthology.orchestra.console.smithi122:power off output: Chassis Power Control: Down/Off
2024-04-18T13:20:14.392 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T13:20:14.407 DEBUG:teuthology.orchestra.console.smithi203:power off output: Chassis Power Control: Down/Off
2024-04-18T13:20:14.407 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T13:20:14.416 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off
2024-04-18T13:20:14.429 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is off
2024-04-18T13:20:14.444 INFO:teuthology.orchestra.console.smithi121:Power off completed
2024-04-18T13:20:14.491 INFO:teuthology.orchestra.console.smithi153:Power off completed
2024-04-18T13:20:14.517 INFO:teuthology.orchestra.console.smithi122:Power off completed
2024-04-18T13:20:14.530 INFO:teuthology.orchestra.console.smithi203:Power off completed
2024-04-18T13:20:14.545 INFO:teuthology.orchestra.console.smithi121:Power on
2024-04-18T13:20:14.545 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-18T13:20:14.569 DEBUG:teuthology.orchestra.console.smithi121:power on output: Chassis Power Control: Up/On
2024-04-18T13:20:14.569 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T13:20:14.590 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is off
2024-04-18T13:20:14.593 INFO:teuthology.orchestra.console.smithi153:Power on
2024-04-18T13:20:14.593 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-18T13:20:14.617 DEBUG:teuthology.orchestra.console.smithi153:power on output: Chassis Power Control: Up/On
2024-04-18T13:20:14.617 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T13:20:14.626 INFO:teuthology.orchestra.console.smithi122:Power on
2024-04-18T13:20:14.627 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-18T13:20:14.635 INFO:teuthology.orchestra.console.smithi203:Power on
2024-04-18T13:20:14.636 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-18T13:20:14.644 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off
2024-04-18T13:20:14.652 DEBUG:teuthology.orchestra.console.smithi122:power on output: Chassis Power Control: Up/On
2024-04-18T13:20:14.652 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T13:20:14.660 DEBUG:teuthology.orchestra.console.smithi203:power on output: Chassis Power Control: Up/On
2024-04-18T13:20:14.661 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T13:20:14.673 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off
2024-04-18T13:20:14.682 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is off
2024-04-18T13:20:18.595 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T13:20:18.645 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T13:20:18.675 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T13:20:18.682 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T13:20:18.705 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is on
2024-04-18T13:20:18.754 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on
2024-04-18T13:20:18.784 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on
2024-04-18T13:20:18.792 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is off
2024-04-18T13:20:18.806 INFO:teuthology.orchestra.console.smithi121:Power on completed
2024-04-18T13:20:18.855 INFO:teuthology.orchestra.console.smithi153:Power on completed
2024-04-18T13:20:18.885 INFO:teuthology.orchestra.console.smithi122:Power on completed
2024-04-18T13:20:18.906 INFO:teuthology.provision.fog.smithi121:Waiting for deploy to finish
2024-04-18T13:20:18.956 INFO:teuthology.provision.fog.smithi153:Waiting for deploy to finish
2024-04-18T13:20:18.986 INFO:teuthology.provision.fog.smithi122:Waiting for deploy to finish
2024-04-18T13:20:22.795 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T13:20:22.905 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on
2024-04-18T13:20:23.005 INFO:teuthology.orchestra.console.smithi203:Power on completed
2024-04-18T13:20:23.106 INFO:teuthology.provision.fog.smithi203:Waiting for deploy to finish
2024-04-18T13:22:50.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:05.188 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:05.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:08.254 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121
2024-04-18T13:23:08.254 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2024-04-18T13:23:09.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:12.318 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2024-04-18T13:23:14.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:14.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:17.310 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121
2024-04-18T13:23:17.311 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2024-04-18T13:23:18.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:21.374 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2024-04-18T13:23:23.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:23.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:26.366 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121
2024-04-18T13:23:26.366 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2024-04-18T13:23:27.376 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:30.430 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2024-04-18T13:23:32.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:32.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:35.422 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121
2024-04-18T13:23:35.423 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2024-04-18T13:23:36.431 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:39.486 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2024-04-18T13:23:41.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:41.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:44.478 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121
2024-04-18T13:23:44.478 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2024-04-18T13:23:45.487 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:48.542 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2024-04-18T13:23:50.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:50.481 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:53.534 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2024-04-18T13:23:54.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:23:57.598 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2024-04-18T13:23:59.535 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:24:02.590 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2024-04-18T13:24:03.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:24:06.654 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2024-04-18T13:24:08.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:24:12.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:24:20.802 ERROR:teuthology.orchestra.connection:Error authenticating with smithi122.front.sepia.ceph.com: AuthenticationException('Authentication timeout.')
2024-04-18T13:24:40.355 ERROR:teuthology.orchestra.connection:Error authenticating with smithi122.front.sepia.ceph.com: BadHostKeyException('smithi122.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:kwzUJuR59ho8SddpDigqx4PUGH3m+sQFbeq6Asgamws), PKey(alg=ED25519, bits=256, fp=SHA256:QvIysNpmYLGySxfVOey78pneqgbQHKtflYjB+uKxXEM)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi122.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO2Kubeav8K8ILFshllk2Mb/byIyOh9RVwLa1sU2623JfLBUy5MggstWFDTtXVizSoMg1UYxOK2/chVdXsDpxVo=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIIQw3UyOxGQGWe7kXarLDYkZyeCSTsAVQ4Y/MLS0xzjv'
2024-04-18T13:24:47.375 ERROR:teuthology.orchestra.connection:Error authenticating with smithi122.front.sepia.ceph.com: BadHostKeyException('smithi122.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:kwzUJuR59ho8SddpDigqx4PUGH3m+sQFbeq6Asgamws), PKey(alg=ED25519, bits=256, fp=SHA256:QvIysNpmYLGySxfVOey78pneqgbQHKtflYjB+uKxXEM)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi122.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO2Kubeav8K8ILFshllk2Mb/byIyOh9RVwLa1sU2623JfLBUy5MggstWFDTtXVizSoMg1UYxOK2/chVdXsDpxVo=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIIQw3UyOxGQGWe7kXarLDYkZyeCSTsAVQ4Y/MLS0xzjv'
2024-04-18T13:24:50.482 WARNING:teuthology.provision.fog.smithi121:timed out
2024-04-18T13:24:56.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:24:56.763 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-18T13:24:56.764 DEBUG:teuthology.orchestra.run.smithi121:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-18T13:24:57.100 INFO:teuthology.provision.fog.smithi121:Node is ready
2024-04-18T13:24:57.100 DEBUG:teuthology.orchestra.run.smithi121:> hostname
2024-04-18T13:24:57.114 INFO:teuthology.orchestra.run.smithi121.stdout:smithi121.front.sepia.ceph.com
2024-04-18T13:24:57.115 DEBUG:teuthology.orchestra.run.smithi121:> grep smithi121.front.sepia.ceph.com /etc/hosts
2024-04-18T13:24:57.170 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.15.121 smithi121.front.sepia.ceph.com smithi121
2024-04-18T13:24:57.170 DEBUG:teuthology.orchestra.run.smithi121:> sudo hostname smithi121
2024-04-18T13:24:57.271 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i -e 's/smithi121.front.sepia.ceph.com/smithi121/g' /etc/hosts
2024-04-18T13:24:57.299 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i -e 's/172.21.15.121/172.21.15.121/g' /etc/hosts
2024-04-18T13:24:57.365 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i -e 's/smithi121.front.sepia.ceph.com/smithi121/g' /etc/hostname
2024-04-18T13:24:57.394 ERROR:teuthology.orchestra.connection:Error authenticating with smithi122.front.sepia.ceph.com: BadHostKeyException('smithi122.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:kwzUJuR59ho8SddpDigqx4PUGH3m+sQFbeq6Asgamws), PKey(alg=ED25519, bits=256, fp=SHA256:QvIysNpmYLGySxfVOey78pneqgbQHKtflYjB+uKxXEM)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi122.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO2Kubeav8K8ILFshllk2Mb/byIyOh9RVwLa1sU2623JfLBUy5MggstWFDTtXVizSoMg1UYxOK2/chVdXsDpxVo=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIIQw3UyOxGQGWe7kXarLDYkZyeCSTsAVQ4Y/MLS0xzjv'
2024-04-18T13:24:57.433 DEBUG:teuthology.orchestra.run.smithi121:> sudo hostname smithi121
2024-04-18T13:24:57.499 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release
2024-04-18T13:24:57.553 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream"
2024-04-18T13:24:57.553 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="9"
2024-04-18T13:24:57.554 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos"
2024-04-18T13:24:57.554 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora"
2024-04-18T13:24:57.554 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="9"
2024-04-18T13:24:57.554 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el9"
2024-04-18T13:24:57.554 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-18T13:24:57.554 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31"
2024-04-18T13:24:57.554 INFO:teuthology.orchestra.run.smithi121.stdout:LOGO="fedora-logo-icon"
2024-04-18T13:24:57.554 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-18T13:24:57.554 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/"
2024-04-18T13:24:57.554 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T13:24:57.554 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-18T13:24:57.554 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-18T13:24:57.555 INFO:teuthology.provision.fog.smithi121:Deploy complete!
2024-04-18T13:24:57.556 DEBUG:teuthology.parallel:result is None
2024-04-18T13:25:02.878 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2024-04-18T13:25:08.594 WARNING:teuthology.provision.fog.smithi153:timed out
2024-04-18T13:25:10.414 ERROR:teuthology.orchestra.connection:Error authenticating with smithi122.front.sepia.ceph.com: BadHostKeyException('smithi122.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:kwzUJuR59ho8SddpDigqx4PUGH3m+sQFbeq6Asgamws), PKey(alg=ED25519, bits=256, fp=SHA256:QvIysNpmYLGySxfVOey78pneqgbQHKtflYjB+uKxXEM)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi122.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO2Kubeav8K8ILFshllk2Mb/byIyOh9RVwLa1sU2623JfLBUy5MggstWFDTtXVizSoMg1UYxOK2/chVdXsDpxVo=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIIQw3UyOxGQGWe7kXarLDYkZyeCSTsAVQ4Y/MLS0xzjv'
2024-04-18T13:25:12.658 WARNING:teuthology.provision.fog.smithi203:timed out
2024-04-18T13:25:14.595 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:25:14.881 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-18T13:25:14.881 DEBUG:teuthology.orchestra.run.smithi153:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-18T13:25:15.213 INFO:teuthology.provision.fog.smithi153:Node is ready
2024-04-18T13:25:15.213 DEBUG:teuthology.orchestra.run.smithi153:> hostname
2024-04-18T13:25:15.228 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153.front.sepia.ceph.com
2024-04-18T13:25:15.228 DEBUG:teuthology.orchestra.run.smithi153:> grep smithi153.front.sepia.ceph.com /etc/hosts
2024-04-18T13:25:15.282 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153 smithi153.front.sepia.ceph.com smithi153
2024-04-18T13:25:15.283 DEBUG:teuthology.orchestra.run.smithi153:> sudo hostname smithi153
2024-04-18T13:25:15.389 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i -e 's/smithi153.front.sepia.ceph.com/smithi153/g' /etc/hosts
2024-04-18T13:25:15.416 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i -e 's/172.21.15.153/172.21.15.153/g' /etc/hosts
2024-04-18T13:25:15.484 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i -e 's/smithi153.front.sepia.ceph.com/smithi153/g' /etc/hostname
2024-04-18T13:25:15.551 DEBUG:teuthology.orchestra.run.smithi153:> sudo hostname smithi153
2024-04-18T13:25:15.618 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release
2024-04-18T13:25:15.673 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream"
2024-04-18T13:25:15.673 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="9"
2024-04-18T13:25:15.673 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos"
2024-04-18T13:25:15.673 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora"
2024-04-18T13:25:15.673 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="9"
2024-04-18T13:25:15.673 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el9"
2024-04-18T13:25:15.673 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-18T13:25:15.674 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31"
2024-04-18T13:25:15.674 INFO:teuthology.orchestra.run.smithi153.stdout:LOGO="fedora-logo-icon"
2024-04-18T13:25:15.674 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-18T13:25:15.674 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/"
2024-04-18T13:25:15.674 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T13:25:15.674 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-18T13:25:15.674 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-18T13:25:15.674 INFO:teuthology.provision.fog.smithi153:Deploy complete!
2024-04-18T13:25:15.675 DEBUG:teuthology.parallel:result is None
2024-04-18T13:25:18.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:25:18.913 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-18T13:25:18.913 DEBUG:teuthology.orchestra.run.smithi203:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-18T13:25:19.259 INFO:teuthology.provision.fog.smithi203:Node is ready
2024-04-18T13:25:19.260 DEBUG:teuthology.orchestra.run.smithi203:> hostname
2024-04-18T13:25:19.275 INFO:teuthology.orchestra.run.smithi203.stdout:smithi203.front.sepia.ceph.com
2024-04-18T13:25:19.275 DEBUG:teuthology.orchestra.run.smithi203:> grep smithi203.front.sepia.ceph.com /etc/hosts
2024-04-18T13:25:19.331 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.15.203 smithi203.front.sepia.ceph.com smithi203
2024-04-18T13:25:19.332 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname smithi203
2024-04-18T13:25:19.434 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i -e 's/smithi203.front.sepia.ceph.com/smithi203/g' /etc/hosts
2024-04-18T13:25:19.462 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i -e 's/172.21.15.203/172.21.15.203/g' /etc/hosts
2024-04-18T13:25:19.530 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i -e 's/smithi203.front.sepia.ceph.com/smithi203/g' /etc/hostname
2024-04-18T13:25:19.598 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname smithi203
2024-04-18T13:25:19.664 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release
2024-04-18T13:25:19.718 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="CentOS Stream"
2024-04-18T13:25:19.719 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="9"
2024-04-18T13:25:19.719 INFO:teuthology.orchestra.run.smithi203.stdout:ID="centos"
2024-04-18T13:25:19.719 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="rhel fedora"
2024-04-18T13:25:19.719 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="9"
2024-04-18T13:25:19.719 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el9"
2024-04-18T13:25:19.719 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-18T13:25:19.719 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31"
2024-04-18T13:25:19.719 INFO:teuthology.orchestra.run.smithi203.stdout:LOGO="fedora-logo-icon"
2024-04-18T13:25:19.719 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-18T13:25:19.719 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://centos.org/"
2024-04-18T13:25:19.719 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T13:25:19.719 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-18T13:25:19.719 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-18T13:25:19.720 INFO:teuthology.provision.fog.smithi203:Deploy complete!
2024-04-18T13:25:19.720 DEBUG:teuthology.parallel:result is None
2024-04-18T13:25:26.433 ERROR:teuthology.orchestra.connection:Error authenticating with smithi122.front.sepia.ceph.com: BadHostKeyException('smithi122.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:kwzUJuR59ho8SddpDigqx4PUGH3m+sQFbeq6Asgamws), PKey(alg=ED25519, bits=256, fp=SHA256:QvIysNpmYLGySxfVOey78pneqgbQHKtflYjB+uKxXEM)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi122.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO2Kubeav8K8ILFshllk2Mb/byIyOh9RVwLa1sU2623JfLBUy5MggstWFDTtXVizSoMg1UYxOK2/chVdXsDpxVo=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIIQw3UyOxGQGWe7kXarLDYkZyeCSTsAVQ4Y/MLS0xzjv'
2024-04-18T13:25:45.452 ERROR:teuthology.orchestra.connection:Error authenticating with smithi122.front.sepia.ceph.com: BadHostKeyException('smithi122.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:kwzUJuR59ho8SddpDigqx4PUGH3m+sQFbeq6Asgamws), PKey(alg=ED25519, bits=256, fp=SHA256:QvIysNpmYLGySxfVOey78pneqgbQHKtflYjB+uKxXEM)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi122.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO2Kubeav8K8ILFshllk2Mb/byIyOh9RVwLa1sU2623JfLBUy5MggstWFDTtXVizSoMg1UYxOK2/chVdXsDpxVo=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIIQw3UyOxGQGWe7kXarLDYkZyeCSTsAVQ4Y/MLS0xzjv'
2024-04-18T13:26:07.471 ERROR:teuthology.orchestra.connection:Error authenticating with smithi122.front.sepia.ceph.com: BadHostKeyException('smithi122.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:kwzUJuR59ho8SddpDigqx4PUGH3m+sQFbeq6Asgamws), PKey(alg=ED25519, bits=256, fp=SHA256:QvIysNpmYLGySxfVOey78pneqgbQHKtflYjB+uKxXEM)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi122.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO2Kubeav8K8ILFshllk2Mb/byIyOh9RVwLa1sU2623JfLBUy5MggstWFDTtXVizSoMg1UYxOK2/chVdXsDpxVo=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIIQw3UyOxGQGWe7kXarLDYkZyeCSTsAVQ4Y/MLS0xzjv'
2024-04-18T13:26:32.491 ERROR:teuthology.orchestra.connection:Error authenticating with smithi122.front.sepia.ceph.com: BadHostKeyException('smithi122.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:kwzUJuR59ho8SddpDigqx4PUGH3m+sQFbeq6Asgamws), PKey(alg=ED25519, bits=256, fp=SHA256:QvIysNpmYLGySxfVOey78pneqgbQHKtflYjB+uKxXEM)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi122.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO2Kubeav8K8ILFshllk2Mb/byIyOh9RVwLa1sU2623JfLBUy5MggstWFDTtXVizSoMg1UYxOK2/chVdXsDpxVo=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIIQw3UyOxGQGWe7kXarLDYkZyeCSTsAVQ4Y/MLS0xzjv'
2024-04-18T13:27:00.511 ERROR:teuthology.orchestra.connection:Error authenticating with smithi122.front.sepia.ceph.com: BadHostKeyException('smithi122.front.sepia.ceph.com', PKey(alg=ECDSA, bits=256, fp=SHA256:kwzUJuR59ho8SddpDigqx4PUGH3m+sQFbeq6Asgamws), PKey(alg=ED25519, bits=256, fp=SHA256:QvIysNpmYLGySxfVOey78pneqgbQHKtflYjB+uKxXEM)) (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect
    raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi122.front.sepia.ceph.com' does not match: got 'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO2Kubeav8K8ILFshllk2Mb/byIyOh9RVwLa1sU2623JfLBUy5MggstWFDTtXVizSoMg1UYxOK2/chVdXsDpxVo=', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIIQw3UyOxGQGWe7kXarLDYkZyeCSTsAVQ4Y/MLS0xzjv'
2024-04-18T13:27:00.512 WARNING:teuthology.provision.fog.smithi122:'connect to smithi122.front.sepia.ceph.com' reached maximum tries (11) after waiting for 145 seconds
2024-04-18T13:27:06.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:27:06.736 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-18T13:27:06.736 DEBUG:teuthology.orchestra.run.smithi122:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-18T13:27:07.089 INFO:teuthology.provision.fog.smithi122:Node is ready
2024-04-18T13:27:07.089 DEBUG:teuthology.orchestra.run.smithi122:> hostname
2024-04-18T13:27:07.104 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122.front.sepia.ceph.com
2024-04-18T13:27:07.104 DEBUG:teuthology.orchestra.run.smithi122:> grep smithi122.front.sepia.ceph.com /etc/hosts
2024-04-18T13:27:07.158 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122 smithi122.front.sepia.ceph.com smithi122
2024-04-18T13:27:07.159 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname smithi122
2024-04-18T13:27:07.302 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/smithi122.front.sepia.ceph.com/smithi122/g' /etc/hosts
2024-04-18T13:27:07.329 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/172.21.15.122/172.21.15.122/g' /etc/hosts
2024-04-18T13:27:07.396 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/smithi122.front.sepia.ceph.com/smithi122/g' /etc/hostname
2024-04-18T13:27:07.463 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname smithi122
2024-04-18T13:27:07.529 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release
2024-04-18T13:27:07.583 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream"
2024-04-18T13:27:07.583 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9"
2024-04-18T13:27:07.584 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos"
2024-04-18T13:27:07.584 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora"
2024-04-18T13:27:07.584 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9"
2024-04-18T13:27:07.584 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9"
2024-04-18T13:27:07.584 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-18T13:27:07.584 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31"
2024-04-18T13:27:07.584 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon"
2024-04-18T13:27:07.584 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-18T13:27:07.584 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/"
2024-04-18T13:27:07.584 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T13:27:07.584 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-18T13:27:07.584 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-18T13:27:07.584 INFO:teuthology.provision.fog.smithi122:Deploy complete!
2024-04-18T13:27:07.585 DEBUG:teuthology.parallel:result is None
2024-04-18T13:27:07.745 ERROR:teuthology.misc:smithi122.front.sepia.ceph.com: Connection closed by remote host
2024-04-18T13:27:07.833 INFO:teuthology.lock.ops:Checking smithi121.front.sepia.ceph.com
2024-04-18T13:27:07.833 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-18T13:27:07.855 INFO:teuthology.lock.ops:Checking smithi122.front.sepia.ceph.com
2024-04-18T13:27:07.855 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-18T13:27:07.871 INFO:teuthology.lock.ops:Checking smithi153.front.sepia.ceph.com
2024-04-18T13:27:07.872 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-18T13:27:07.888 INFO:teuthology.lock.ops:Checking smithi203.front.sepia.ceph.com
2024-04-18T13:27:07.888 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-18T13:27:07.906 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: set os type and version on server
2024-04-18T13:27:07.906 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com'
2024-04-18T13:27:07.907 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:27:07.992 DEBUG:teuthology.orchestra.run.smithi121:> true
2024-04-18T13:27:08.118 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com'
2024-04-18T13:27:08.118 DEBUG:teuthology.orchestra.run.smithi121:> uname -m
2024-04-18T13:27:08.172 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64
2024-04-18T13:27:08.173 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release
2024-04-18T13:27:08.227 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream"
2024-04-18T13:27:08.227 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="9"
2024-04-18T13:27:08.227 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos"
2024-04-18T13:27:08.227 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora"
2024-04-18T13:27:08.227 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="9"
2024-04-18T13:27:08.227 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el9"
2024-04-18T13:27:08.228 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-18T13:27:08.228 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31"
2024-04-18T13:27:08.228 INFO:teuthology.orchestra.run.smithi121.stdout:LOGO="fedora-logo-icon"
2024-04-18T13:27:08.228 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-18T13:27:08.228 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/"
2024-04-18T13:27:08.228 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T13:27:08.228 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-18T13:27:08.228 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-18T13:27:08.228 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server
2024-04-18T13:27:08.255 INFO:teuthology.lock.ops:Updating [smithi122.front.sepia.ceph.com]: set os type and version on server
2024-04-18T13:27:08.255 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com'
2024-04-18T13:27:08.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:27:08.340 DEBUG:teuthology.orchestra.run.smithi122:> true
2024-04-18T13:27:08.362 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com'
2024-04-18T13:27:08.362 DEBUG:teuthology.orchestra.run.smithi122:> uname -m
2024-04-18T13:27:08.416 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64
2024-04-18T13:27:08.416 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release
2024-04-18T13:27:08.470 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream"
2024-04-18T13:27:08.470 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9"
2024-04-18T13:27:08.470 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos"
2024-04-18T13:27:08.470 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora"
2024-04-18T13:27:08.470 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9"
2024-04-18T13:27:08.470 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9"
2024-04-18T13:27:08.470 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-18T13:27:08.470 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31"
2024-04-18T13:27:08.470 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon"
2024-04-18T13:27:08.470 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-18T13:27:08.470 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/"
2024-04-18T13:27:08.470 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T13:27:08.471 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-18T13:27:08.471 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-18T13:27:08.471 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server
2024-04-18T13:27:08.489 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: set os type and version on server
2024-04-18T13:27:08.489 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com'
2024-04-18T13:27:08.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:27:08.572 DEBUG:teuthology.orchestra.run.smithi153:> true
2024-04-18T13:27:08.740 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com'
2024-04-18T13:27:08.740 DEBUG:teuthology.orchestra.run.smithi153:> uname -m
2024-04-18T13:27:08.795 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64
2024-04-18T13:27:08.795 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release
2024-04-18T13:27:08.850 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream"
2024-04-18T13:27:08.850 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="9"
2024-04-18T13:27:08.850 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos"
2024-04-18T13:27:08.850 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora"
2024-04-18T13:27:08.850 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="9"
2024-04-18T13:27:08.850 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el9"
2024-04-18T13:27:08.850 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-18T13:27:08.850 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31"
2024-04-18T13:27:08.850 INFO:teuthology.orchestra.run.smithi153.stdout:LOGO="fedora-logo-icon"
2024-04-18T13:27:08.850 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-18T13:27:08.851 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/"
2024-04-18T13:27:08.851 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T13:27:08.851 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-18T13:27:08.851 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-18T13:27:08.851 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server
2024-04-18T13:27:08.872 INFO:teuthology.lock.ops:Updating [smithi203.front.sepia.ceph.com]: set os type and version on server
2024-04-18T13:27:08.872 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com'
2024-04-18T13:27:08.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T13:27:08.955 DEBUG:teuthology.orchestra.run.smithi203:> true
2024-04-18T13:27:09.128 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com'
2024-04-18T13:27:09.128 DEBUG:teuthology.orchestra.run.smithi203:> uname -m
2024-04-18T13:27:09.182 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64
2024-04-18T13:27:09.183 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release
2024-04-18T13:27:09.237 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="CentOS Stream"
2024-04-18T13:27:09.237 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="9"
2024-04-18T13:27:09.238 INFO:teuthology.orchestra.run.smithi203.stdout:ID="centos"
2024-04-18T13:27:09.238 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="rhel fedora"
2024-04-18T13:27:09.238 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="9"
2024-04-18T13:27:09.238 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el9"
2024-04-18T13:27:09.238 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-18T13:27:09.238 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31"
2024-04-18T13:27:09.238 INFO:teuthology.orchestra.run.smithi203.stdout:LOGO="fedora-logo-icon"
2024-04-18T13:27:09.238 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-18T13:27:09.238 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://centos.org/"
2024-04-18T13:27:09.238 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T13:27:09.238 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-18T13:27:09.238 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-18T13:27:09.239 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server
2024-04-18T13:27:09.263 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-18T13:27:09.353 INFO:teuthology.dispatcher.supervisor:Running job 7662060
2024-04-18T13:27:09.353 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/bin/teuthology -v --owner scheduled_nmordech@teuthology --archive /home/teuthworker/archive/nmordech-2024-04-18_13:06:18-rados:thrash-erasure-code-crush-4-nodes-main-distro-default-smithi/7662060 --name nmordech-2024-04-18_13:06:18-rados:thrash-erasure-code-crush-4-nodes-main-distro-default-smithi --description rados:thrash-erasure-code-crush-4-nodes/{arch/x86_64 ceph mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=8-m=6-crush} -- /home/teuthworker/archive/nmordech-2024-04-18_13:06:18-rados:thrash-erasure-code-crush-4-nodes-main-distro-default-smithi/7662060/orig.config.yaml
2024-04-18T13:27:09.356 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nmordech-2024-04-18_13:06:18-rados:thrash-erasure-code-crush-4-nodes-main-distro-default-smithi/7662060
2024-04-18T13:27:09.357 INFO:teuthology.dispatcher.supervisor:Job PID: 3895281
2024-04-18T13:27:09.357 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-04-18T13:29:09.396 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-18T13:31:09.567 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-18T13:33:09.730 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-18T13:35:09.873 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-18T13:35:09.942 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1