2024-01-27T03:05:55.744 INFO:root:teuthology version: 0.0.1.dev251+gc856e0f 2024-01-27T03:05:55.745 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-27T03:05:55.986 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff4be493c10>, signals=[15]) 2024-01-27T03:05:55.986 INFO:teuthology.lock.ops:Start node 'smithi039.front.sepia.ceph.com' reimaging 2024-01-27T03:05:55.986 INFO:teuthology.lock.ops:Updating [smithi039.front.sepia.ceph.com]: reset os type and version on server 2024-01-27T03:05:55.986 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2024-01-27T03:05:56.008 INFO:teuthology.lock.ops:Start node 'smithi112.front.sepia.ceph.com' reimaging 2024-01-27T03:05:56.008 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: reset os type and version on server 2024-01-27T03:05:56.008 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-01-27T03:05:56.032 INFO:teuthology.lock.ops:Start node 'smithi125.front.sepia.ceph.com' reimaging 2024-01-27T03:05:56.033 INFO:teuthology.lock.ops:Updating [smithi125.front.sepia.ceph.com]: reset os type and version on server 2024-01-27T03:05:56.033 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server 2024-01-27T03:05:56.057 INFO:teuthology.lock.ops:Start node 'smithi151.front.sepia.ceph.com' reimaging 2024-01-27T03:05:56.057 INFO:teuthology.lock.ops:Updating [smithi151.front.sepia.ceph.com]: reset os type and version on server 2024-01-27T03:05:56.057 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2024-01-27T03:05:56.082 INFO:teuthology.lock.ops:Start node 'smithi176.front.sepia.ceph.com' reimaging 2024-01-27T03:05:56.082 INFO:teuthology.lock.ops:Updating [smithi176.front.sepia.ceph.com]: reset os type and version on server 2024-01-27T03:05:56.082 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2024-01-27T03:05:56.148 DEBUG:teuthology.provision.fog.smithi112:Requesting image smithi_ubuntu_20.04 (ID 2) 2024-01-27T03:05:56.151 DEBUG:teuthology.provision.fog.smithi039:Requesting image smithi_ubuntu_20.04 (ID 2) 2024-01-27T03:05:56.169 DEBUG:teuthology.provision.fog.smithi125:Requesting image smithi_ubuntu_20.04 (ID 2) 2024-01-27T03:05:56.203 INFO:teuthology.provision.fog.smithi039:Scheduling deploy of ubuntu 20.04 2024-01-27T03:05:56.206 INFO:teuthology.provision.fog.smithi112:Scheduling deploy of ubuntu 20.04 2024-01-27T03:05:56.209 DEBUG:teuthology.provision.fog.smithi151:Requesting image smithi_ubuntu_20.04 (ID 2) 2024-01-27T03:05:56.219 DEBUG:teuthology.provision.fog.smithi176:Requesting image smithi_ubuntu_20.04 (ID 2) 2024-01-27T03:05:56.222 INFO:teuthology.provision.fog.smithi125:Scheduling deploy of ubuntu 20.04 2024-01-27T03:05:56.255 INFO:teuthology.provision.fog.smithi151:Scheduling deploy of ubuntu 20.04 2024-01-27T03:05:56.279 INFO:teuthology.provision.fog.smithi176:Scheduling deploy of ubuntu 20.04 2024-01-27T03:05:56.481 INFO:teuthology.orchestra.console.smithi125:Power off 2024-01-27T03:05:56.482 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-27T03:05:56.500 INFO:teuthology.orchestra.console.smithi112:Power off 2024-01-27T03:05:56.500 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-27T03:05:56.511 INFO:teuthology.orchestra.console.smithi039:Power off 2024-01-27T03:05:56.512 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-27T03:05:56.526 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off 2024-01-27T03:05:56.526 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:05:56.534 DEBUG:teuthology.orchestra.console.smithi125:power off output: Chassis Power Control: Down/Off 2024-01-27T03:05:56.534 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:05:56.543 DEBUG:teuthology.orchestra.console.smithi039:power off output: Chassis Power Control: Down/Off 2024-01-27T03:05:56.543 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:05:56.551 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-01-27T03:05:56.556 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2024-01-27T03:05:56.566 INFO:teuthology.orchestra.console.smithi176:Power off 2024-01-27T03:05:56.567 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-27T03:05:56.579 INFO:teuthology.orchestra.console.smithi151:Power off 2024-01-27T03:05:56.580 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-27T03:05:56.586 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-01-27T03:05:56.591 DEBUG:teuthology.orchestra.console.smithi176:power off output: Chassis Power Control: Down/Off 2024-01-27T03:05:56.591 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:05:56.600 DEBUG:teuthology.orchestra.console.smithi151:power off output: Chassis Power Control: Down/Off 2024-01-27T03:05:56.600 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:05:56.611 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is on 2024-01-27T03:05:56.623 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2024-01-27T03:06:00.556 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:00.564 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:00.587 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:00.613 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:00.624 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:00.666 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-01-27T03:06:00.674 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2024-01-27T03:06:00.697 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-01-27T03:06:00.722 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is on 2024-01-27T03:06:00.733 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2024-01-27T03:06:04.668 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:04.676 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:04.698 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:04.723 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:04.734 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:04.778 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-01-27T03:06:04.787 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2024-01-27T03:06:04.807 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-01-27T03:06:04.832 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is off 2024-01-27T03:06:04.843 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2024-01-27T03:06:04.879 INFO:teuthology.orchestra.console.smithi112:Power off completed 2024-01-27T03:06:04.934 INFO:teuthology.orchestra.console.smithi176:Power off completed 2024-01-27T03:06:04.979 INFO:teuthology.orchestra.console.smithi112:Power on 2024-01-27T03:06:04.980 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-27T03:06:05.004 DEBUG:teuthology.orchestra.console.smithi112:power on output: Chassis Power Control: Up/On 2024-01-27T03:06:05.005 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:05.026 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-01-27T03:06:05.035 INFO:teuthology.orchestra.console.smithi176:Power on 2024-01-27T03:06:05.035 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-27T03:06:05.056 DEBUG:teuthology.orchestra.console.smithi176:power on output: Chassis Power Control: Up/On 2024-01-27T03:06:05.056 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:05.076 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is off 2024-01-27T03:06:08.792 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:08.808 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:08.844 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:08.900 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off 2024-01-27T03:06:08.917 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is off 2024-01-27T03:06:08.953 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is off 2024-01-27T03:06:09.002 INFO:teuthology.orchestra.console.smithi125:Power off completed 2024-01-27T03:06:09.018 INFO:teuthology.orchestra.console.smithi039:Power off completed 2024-01-27T03:06:09.028 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:09.054 INFO:teuthology.orchestra.console.smithi151:Power off completed 2024-01-27T03:06:09.077 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:09.103 INFO:teuthology.orchestra.console.smithi125:Power on 2024-01-27T03:06:09.103 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-27T03:06:09.119 INFO:teuthology.orchestra.console.smithi039:Power on 2024-01-27T03:06:09.119 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-27T03:06:09.128 DEBUG:teuthology.orchestra.console.smithi125:power on output: Chassis Power Control: Up/On 2024-01-27T03:06:09.128 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:09.138 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-01-27T03:06:09.141 DEBUG:teuthology.orchestra.console.smithi039:power on output: Chassis Power Control: Up/On 2024-01-27T03:06:09.141 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:09.148 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off 2024-01-27T03:06:09.155 INFO:teuthology.orchestra.console.smithi151:Power on 2024-01-27T03:06:09.155 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-27T03:06:09.163 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is off 2024-01-27T03:06:09.175 DEBUG:teuthology.orchestra.console.smithi151:power on output: Chassis Power Control: Up/On 2024-01-27T03:06:09.175 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:09.187 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is off 2024-01-27T03:06:09.199 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is off 2024-01-27T03:06:13.140 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:13.149 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:13.164 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:13.188 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:13.199 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:13.250 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-01-27T03:06:13.258 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off 2024-01-27T03:06:13.272 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-01-27T03:06:13.307 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2024-01-27T03:06:13.351 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is on 2024-01-27T03:06:13.351 INFO:teuthology.orchestra.console.smithi112:Power on completed 2024-01-27T03:06:13.373 INFO:teuthology.orchestra.console.smithi039:Power on completed 2024-01-27T03:06:13.408 INFO:teuthology.orchestra.console.smithi151:Power on completed 2024-01-27T03:06:13.451 INFO:teuthology.orchestra.console.smithi176:Power on completed 2024-01-27T03:06:13.452 INFO:teuthology.provision.fog.smithi112:Waiting for deploy to finish 2024-01-27T03:06:13.474 INFO:teuthology.provision.fog.smithi039:Waiting for deploy to finish 2024-01-27T03:06:13.509 INFO:teuthology.provision.fog.smithi151:Waiting for deploy to finish 2024-01-27T03:06:13.553 INFO:teuthology.provision.fog.smithi176:Waiting for deploy to finish 2024-01-27T03:06:17.260 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:06:17.416 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2024-01-27T03:06:17.517 INFO:teuthology.orchestra.console.smithi125:Power on completed 2024-01-27T03:06:17.617 INFO:teuthology.provision.fog.smithi125:Waiting for deploy to finish 2024-01-27T03:09:44.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:09:44.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:09:47.872 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-01-27T03:09:48.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:09:51.868 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2024-01-27T03:09:53.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:09:56.924 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-01-27T03:09:57.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:00.924 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2024-01-27T03:10:02.925 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:05.980 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-01-27T03:10:06.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:09.980 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2024-01-27T03:10:11.981 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:14.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:15.040 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-01-27T03:10:15.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:17.980 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-01-27T03:10:19.036 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2024-01-27T03:10:21.041 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:23.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:24.095 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-01-27T03:10:25.037 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:27.036 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-01-27T03:10:28.092 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2024-01-27T03:10:29.994 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:30.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:33.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:33.052 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:10:33.148 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-01-27T03:10:34.093 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:36.092 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-01-27T03:10:37.152 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2024-01-27T03:10:39.053 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:39.149 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:42.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:42.112 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:10:42.208 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-01-27T03:10:43.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:45.014 ERROR:teuthology.orchestra.connection:Error authenticating with smithi176.front.sepia.ceph.com: AuthenticationException('Authentication timeout.') 2024-01-27T03:10:45.148 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-01-27T03:10:46.236 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2024-01-27T03:10:48.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:48.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:51.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:51.164 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:10:52.237 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:10:54.204 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-01-27T03:10:54.304 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-01-27T03:10:57.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:11:00.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:11:00.224 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:11:00.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:11:03.260 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-01-27T03:11:03.356 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-01-27T03:11:06.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:11:09.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:11:09.276 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:11:09.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:11:12.316 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-01-27T03:11:12.412 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-01-27T03:11:15.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:11:18.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:11:18.332 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:11:18.413 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:11:19.114 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-27T03:11:19.114 DEBUG:teuthology.orchestra.run.smithi112:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-27T03:11:20.747 ERROR:teuthology.orchestra.connection:Error authenticating with smithi176.front.sepia.ceph.com: BadHostKeyException('smithi176.front.sepia.ceph.com', PKey(alg=ED25519, bits=256, fp=SHA256:haDEfp1JxKh2GOA4mqLcwYgEUzpaY2OZo07qQ7b78Tc), PKey(alg=ED25519, bits=256, fp=SHA256:ERAcbWR5OrPhLu6MqwwEFmPsr2hndQyq9svQk2eAB3g)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi176.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIHrTlCbDexE70YRUUQfEzovsts4mEcbY5xTBZ1B3MPKZ' 2024-01-27T03:11:24.333 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:11:27.388 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:11:27.781 ERROR:teuthology.orchestra.connection:Error authenticating with smithi176.front.sepia.ceph.com: BadHostKeyException('smithi176.front.sepia.ceph.com', PKey(alg=ED25519, bits=256, fp=SHA256:haDEfp1JxKh2GOA4mqLcwYgEUzpaY2OZo07qQ7b78Tc), PKey(alg=ED25519, bits=256, fp=SHA256:ERAcbWR5OrPhLu6MqwwEFmPsr2hndQyq9svQk2eAB3g)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi176.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIHrTlCbDexE70YRUUQfEzovsts4mEcbY5xTBZ1B3MPKZ' 2024-01-27T03:11:33.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:11:35.996 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-01-27T03:11:36.448 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:11:36.764 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-01-27T03:11:37.814 ERROR:teuthology.orchestra.connection:Error authenticating with smithi176.front.sepia.ceph.com: BadHostKeyException('smithi176.front.sepia.ceph.com', PKey(alg=ED25519, bits=256, fp=SHA256:haDEfp1JxKh2GOA4mqLcwYgEUzpaY2OZo07qQ7b78Tc), PKey(alg=ED25519, bits=256, fp=SHA256:ERAcbWR5OrPhLu6MqwwEFmPsr2hndQyq9svQk2eAB3g)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi176.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIHrTlCbDexE70YRUUQfEzovsts4mEcbY5xTBZ1B3MPKZ' 2024-01-27T03:11:40.350 INFO:teuthology.provision.fog.smithi112:Node is ready 2024-01-27T03:11:40.350 DEBUG:teuthology.orchestra.run.smithi112:> hostname 2024-01-27T03:11:40.357 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112.front.sepia.ceph.com 2024-01-27T03:11:40.357 DEBUG:teuthology.orchestra.run.smithi112:> grep smithi112.front.sepia.ceph.com /etc/hosts 2024-01-27T03:11:40.404 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112 smithi112.front.sepia.ceph.com smithi112 2024-01-27T03:11:40.404 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname smithi112 2024-01-27T03:11:40.485 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/smithi112.front.sepia.ceph.com/smithi112/g' /etc/hosts 2024-01-27T03:11:40.500 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/172.21.15.112/172.21.15.112/g' /etc/hosts 2024-01-27T03:11:40.557 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/smithi112.front.sepia.ceph.com/smithi112/g' /etc/hostname 2024-01-27T03:11:40.569 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname smithi112 2024-01-27T03:11:40.580 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-01-27T03:11:40.627 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="Ubuntu" 2024-01-27T03:11:40.628 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-01-27T03:11:40.628 INFO:teuthology.orchestra.run.smithi112.stdout:ID=ubuntu 2024-01-27T03:11:40.628 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE=debian 2024-01-27T03:11:40.628 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-01-27T03:11:40.628 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="20.04" 2024-01-27T03:11:40.628 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://www.ubuntu.com/" 2024-01-27T03:11:40.628 INFO:teuthology.orchestra.run.smithi112.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-01-27T03:11:40.628 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-01-27T03:11:40.628 INFO:teuthology.orchestra.run.smithi112.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-01-27T03:11:40.628 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_CODENAME=focal 2024-01-27T03:11:40.629 INFO:teuthology.orchestra.run.smithi112.stdout:UBUNTU_CODENAME=focal 2024-01-27T03:11:40.629 INFO:teuthology.provision.fog.smithi112:Deploy complete! 2024-01-27T03:11:40.630 DEBUG:teuthology.parallel:result is None 2024-01-27T03:11:42.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:11:42.765 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:11:45.500 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:11:45.820 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-01-27T03:11:50.847 ERROR:teuthology.orchestra.connection:Error authenticating with smithi176.front.sepia.ceph.com: BadHostKeyException('smithi176.front.sepia.ceph.com', PKey(alg=ED25519, bits=256, fp=SHA256:haDEfp1JxKh2GOA4mqLcwYgEUzpaY2OZo07qQ7b78Tc), PKey(alg=ED25519, bits=256, fp=SHA256:ERAcbWR5OrPhLu6MqwwEFmPsr2hndQyq9svQk2eAB3g)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi176.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIHrTlCbDexE70YRUUQfEzovsts4mEcbY5xTBZ1B3MPKZ' 2024-01-27T03:11:51.501 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:11:51.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:11:52.240 WARNING:teuthology.provision.fog.smithi125:timed out 2024-01-27T03:11:53.232 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-27T03:11:53.232 DEBUG:teuthology.orchestra.run.smithi151:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-27T03:11:54.556 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:11:58.241 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:11:58.483 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-27T03:11:58.484 DEBUG:teuthology.orchestra.run.smithi125:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-27T03:11:58.987 INFO:teuthology.provision.fog.smithi125:Node is ready 2024-01-27T03:11:58.987 DEBUG:teuthology.orchestra.run.smithi125:> hostname 2024-01-27T03:11:58.993 INFO:teuthology.orchestra.run.smithi125.stdout:smithi125.front.sepia.ceph.com 2024-01-27T03:11:58.994 DEBUG:teuthology.orchestra.run.smithi125:> grep smithi125.front.sepia.ceph.com /etc/hosts 2024-01-27T03:11:59.001 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.125 smithi125.front.sepia.ceph.com smithi125 2024-01-27T03:11:59.001 DEBUG:teuthology.orchestra.run.smithi125:> sudo hostname smithi125 2024-01-27T03:11:59.135 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/smithi125.front.sepia.ceph.com/smithi125/g' /etc/hosts 2024-01-27T03:11:59.151 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/172.21.15.125/172.21.15.125/g' /etc/hosts 2024-01-27T03:11:59.167 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/smithi125.front.sepia.ceph.com/smithi125/g' /etc/hostname 2024-01-27T03:11:59.182 DEBUG:teuthology.orchestra.run.smithi125:> sudo hostname smithi125 2024-01-27T03:11:59.190 DEBUG:teuthology.orchestra.run.smithi125:> cat /etc/os-release 2024-01-27T03:11:59.195 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="Ubuntu" 2024-01-27T03:11:59.195 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-01-27T03:11:59.196 INFO:teuthology.orchestra.run.smithi125.stdout:ID=ubuntu 2024-01-27T03:11:59.196 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE=debian 2024-01-27T03:11:59.196 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-01-27T03:11:59.196 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="20.04" 2024-01-27T03:11:59.196 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://www.ubuntu.com/" 2024-01-27T03:11:59.196 INFO:teuthology.orchestra.run.smithi125.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-01-27T03:11:59.196 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-01-27T03:11:59.196 INFO:teuthology.orchestra.run.smithi125.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-01-27T03:11:59.196 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_CODENAME=focal 2024-01-27T03:11:59.196 INFO:teuthology.orchestra.run.smithi125.stdout:UBUNTU_CODENAME=focal 2024-01-27T03:11:59.196 INFO:teuthology.provision.fog.smithi125:Deploy complete! 2024-01-27T03:11:59.197 DEBUG:teuthology.parallel:result is None 2024-01-27T03:12:00.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:12:03.612 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:12:06.881 ERROR:teuthology.orchestra.connection:Error authenticating with smithi176.front.sepia.ceph.com: BadHostKeyException('smithi176.front.sepia.ceph.com', PKey(alg=ED25519, bits=256, fp=SHA256:haDEfp1JxKh2GOA4mqLcwYgEUzpaY2OZo07qQ7b78Tc), PKey(alg=ED25519, bits=256, fp=SHA256:ERAcbWR5OrPhLu6MqwwEFmPsr2hndQyq9svQk2eAB3g)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi176.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIHrTlCbDexE70YRUUQfEzovsts4mEcbY5xTBZ1B3MPKZ' 2024-01-27T03:12:09.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:12:12.668 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:12:14.243 INFO:teuthology.provision.fog.smithi151:Node is ready 2024-01-27T03:12:14.243 DEBUG:teuthology.orchestra.run.smithi151:> hostname 2024-01-27T03:12:14.249 INFO:teuthology.orchestra.run.smithi151.stdout:smithi129 2024-01-27T03:12:14.250 DEBUG:teuthology.orchestra.run.smithi151:> grep smithi129 /etc/hosts 2024-01-27T03:12:14.257 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.129 smithi129 smithi129 2024-01-27T03:12:14.257 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname smithi151 2024-01-27T03:12:14.332 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i -e 's/smithi129/smithi151/g' /etc/hosts 2024-01-27T03:12:14.349 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i -e 's/172.21.15.129/172.21.15.151/g' /etc/hosts 2024-01-27T03:12:14.361 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i -e 's/smithi129/smithi151/g' /etc/hostname 2024-01-27T03:12:14.421 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname smithi151 2024-01-27T03:12:14.431 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2024-01-27T03:12:14.436 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="Ubuntu" 2024-01-27T03:12:14.436 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-01-27T03:12:14.437 INFO:teuthology.orchestra.run.smithi151.stdout:ID=ubuntu 2024-01-27T03:12:14.437 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE=debian 2024-01-27T03:12:14.437 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-01-27T03:12:14.437 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="20.04" 2024-01-27T03:12:14.437 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://www.ubuntu.com/" 2024-01-27T03:12:14.437 INFO:teuthology.orchestra.run.smithi151.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-01-27T03:12:14.437 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-01-27T03:12:14.437 INFO:teuthology.orchestra.run.smithi151.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-01-27T03:12:14.437 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_CODENAME=focal 2024-01-27T03:12:14.437 INFO:teuthology.orchestra.run.smithi151.stdout:UBUNTU_CODENAME=focal 2024-01-27T03:12:14.437 INFO:teuthology.provision.fog.smithi151:Deploy complete! 2024-01-27T03:12:14.438 DEBUG:teuthology.parallel:result is None 2024-01-27T03:12:18.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:12:21.724 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:12:25.914 ERROR:teuthology.orchestra.connection:Error authenticating with smithi176.front.sepia.ceph.com: BadHostKeyException('smithi176.front.sepia.ceph.com', PKey(alg=ED25519, bits=256, fp=SHA256:haDEfp1JxKh2GOA4mqLcwYgEUzpaY2OZo07qQ7b78Tc), PKey(alg=ED25519, bits=256, fp=SHA256:ERAcbWR5OrPhLu6MqwwEFmPsr2hndQyq9svQk2eAB3g)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi176.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIHrTlCbDexE70YRUUQfEzovsts4mEcbY5xTBZ1B3MPKZ' 2024-01-27T03:12:27.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:12:30.780 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:12:36.781 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:12:39.836 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:12:45.837 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:12:47.947 ERROR:teuthology.orchestra.connection:Error authenticating with smithi176.front.sepia.ceph.com: BadHostKeyException('smithi176.front.sepia.ceph.com', PKey(alg=ED25519, bits=256, fp=SHA256:haDEfp1JxKh2GOA4mqLcwYgEUzpaY2OZo07qQ7b78Tc), PKey(alg=ED25519, bits=256, fp=SHA256:ERAcbWR5OrPhLu6MqwwEFmPsr2hndQyq9svQk2eAB3g)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi176.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIHrTlCbDexE70YRUUQfEzovsts4mEcbY5xTBZ1B3MPKZ' 2024-01-27T03:12:48.892 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:12:54.894 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:12:57.952 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:13:03.954 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:13:07.004 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:13:12.980 ERROR:teuthology.orchestra.connection:Error authenticating with smithi176.front.sepia.ceph.com: BadHostKeyException('smithi176.front.sepia.ceph.com', PKey(alg=ED25519, bits=256, fp=SHA256:haDEfp1JxKh2GOA4mqLcwYgEUzpaY2OZo07qQ7b78Tc), PKey(alg=ED25519, bits=256, fp=SHA256:ERAcbWR5OrPhLu6MqwwEFmPsr2hndQyq9svQk2eAB3g)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi176.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIHrTlCbDexE70YRUUQfEzovsts4mEcbY5xTBZ1B3MPKZ' 2024-01-27T03:13:13.005 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:13:16.060 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:13:22.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:13:25.120 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:13:31.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:13:34.172 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:13:40.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:13:41.012 ERROR:teuthology.orchestra.connection:Error authenticating with smithi176.front.sepia.ceph.com: BadHostKeyException('smithi176.front.sepia.ceph.com', PKey(alg=ED25519, bits=256, fp=SHA256:haDEfp1JxKh2GOA4mqLcwYgEUzpaY2OZo07qQ7b78Tc), PKey(alg=ED25519, bits=256, fp=SHA256:ERAcbWR5OrPhLu6MqwwEFmPsr2hndQyq9svQk2eAB3g)) (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 468, in connect raise BadHostKeyException(hostname, server_key, our_key) paramiko.ssh_exception.BadHostKeyException: Host key for server 'smithi176.front.sepia.ceph.com' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIHrTlCbDexE70YRUUQfEzovsts4mEcbY5xTBZ1B3MPKZ' 2024-01-27T03:13:41.013 WARNING:teuthology.provision.fog.smithi176:'connect to smithi176.front.sepia.ceph.com' reached maximum tries (11) after waiting for 145 seconds 2024-01-27T03:13:43.228 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:13:47.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:13:47.248 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-27T03:13:47.248 DEBUG:teuthology.orchestra.run.smithi176:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-27T03:13:47.753 INFO:teuthology.provision.fog.smithi176:Node is ready 2024-01-27T03:13:47.754 DEBUG:teuthology.orchestra.run.smithi176:> hostname 2024-01-27T03:13:47.760 INFO:teuthology.orchestra.run.smithi176.stdout:smithi176.front.sepia.ceph.com 2024-01-27T03:13:47.760 DEBUG:teuthology.orchestra.run.smithi176:> grep smithi176.front.sepia.ceph.com /etc/hosts 2024-01-27T03:13:47.809 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176 smithi176.front.sepia.ceph.com smithi176 2024-01-27T03:13:47.809 DEBUG:teuthology.orchestra.run.smithi176:> sudo hostname smithi176 2024-01-27T03:13:47.892 DEBUG:teuthology.orchestra.run.smithi176:> sudo sed -i -e 's/smithi176.front.sepia.ceph.com/smithi176/g' /etc/hosts 2024-01-27T03:13:47.909 DEBUG:teuthology.orchestra.run.smithi176:> sudo sed -i -e 's/172.21.15.176/172.21.15.176/g' /etc/hosts 2024-01-27T03:13:47.919 DEBUG:teuthology.orchestra.run.smithi176:> sudo sed -i -e 's/smithi176.front.sepia.ceph.com/smithi176/g' /etc/hostname 2024-01-27T03:13:47.978 DEBUG:teuthology.orchestra.run.smithi176:> sudo hostname smithi176 2024-01-27T03:13:47.987 DEBUG:teuthology.orchestra.run.smithi176:> cat /etc/os-release 2024-01-27T03:13:47.993 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="Ubuntu" 2024-01-27T03:13:47.993 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-01-27T03:13:47.993 INFO:teuthology.orchestra.run.smithi176.stdout:ID=ubuntu 2024-01-27T03:13:47.993 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE=debian 2024-01-27T03:13:47.993 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-01-27T03:13:47.993 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="20.04" 2024-01-27T03:13:47.994 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://www.ubuntu.com/" 2024-01-27T03:13:47.994 INFO:teuthology.orchestra.run.smithi176.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-01-27T03:13:47.994 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-01-27T03:13:47.994 INFO:teuthology.orchestra.run.smithi176.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-01-27T03:13:47.994 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_CODENAME=focal 2024-01-27T03:13:47.994 INFO:teuthology.orchestra.run.smithi176.stdout:UBUNTU_CODENAME=focal 2024-01-27T03:13:47.994 INFO:teuthology.provision.fog.smithi176:Deploy complete! 2024-01-27T03:13:47.995 DEBUG:teuthology.parallel:result is None 2024-01-27T03:13:49.229 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:13:52.284 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:13:58.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:14:01.340 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:14:07.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:14:10.396 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:14:16.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:14:19.452 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:14:25.453 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:14:28.508 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:14:34.509 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:14:37.564 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:14:43.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:14:46.620 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:14:52.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:14:55.676 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:15:01.677 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:15:04.732 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:15:10.733 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:15:13.788 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:15:19.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:15:22.844 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:15:28.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:15:31.900 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:15:37.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:15:40.956 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:15:46.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:15:50.012 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:15:56.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:15:59.068 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:16:05.070 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:16:08.124 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:16:14.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:16:17.180 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:16:23.181 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:16:26.236 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:16:32.237 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:16:35.296 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:16:41.298 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:16:44.348 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:16:50.349 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:16:53.404 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:16:59.405 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:17:02.460 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:17:08.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:17:11.516 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:17:17.517 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:17:20.572 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:17:26.573 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:17:29.628 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:17:35.630 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:17:38.684 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:17:44.685 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:17:47.740 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:17:53.741 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:17:56.800 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:18:02.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:18:05.852 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:18:11.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:18:14.908 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:18:20.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:18:23.964 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:18:29.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:18:33.020 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:18:39.021 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:18:42.076 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:18:48.078 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:18:51.136 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:18:57.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:19:00.188 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:19:06.189 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:19:09.244 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:19:15.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:19:18.304 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:19:24.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:19:27.356 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:19:33.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:19:36.412 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:19:42.413 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:19:45.468 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:19:51.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:19:54.524 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:20:00.526 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:20:03.580 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:20:09.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:20:12.636 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:20:18.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:20:21.692 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:20:27.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:20:30.752 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:20:36.754 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:20:39.804 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:20:45.805 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:20:48.860 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:20:54.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:20:57.916 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:21:03.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:21:06.972 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:21:12.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:21:16.032 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:21:22.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:21:25.084 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:21:31.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:21:34.140 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:21:40.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:21:43.200 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:21:49.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:21:52.252 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:21:58.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:22:01.308 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:22:07.309 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:22:10.364 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:22:16.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:22:19.420 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:22:25.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:22:28.476 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:22:34.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:22:37.564 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:22:43.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:22:46.652 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:22:52.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:22:55.708 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:23:01.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:23:04.799 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:23:10.805 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:23:13.884 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:23:19.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:23:22.940 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:23:28.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:23:32.028 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:23:38.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:23:41.084 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:23:47.092 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:23:50.172 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:23:56.176 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:23:59.228 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:24:05.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:24:08.284 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:24:14.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:24:17.376 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:24:23.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:24:26.460 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:24:32.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:24:35.552 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:24:41.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:24:44.636 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:24:50.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:24:53.692 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:24:59.696 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:25:02.748 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:25:08.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:25:11.808 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:25:17.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:25:20.892 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:25:26.899 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T03:25:29.980 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-01-27T03:25:29.983 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2024-01-27T03:25:29.984 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-27T03:25:34.047 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=06bb3ce96fed4318b8544690e37c5b42 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2024-01-27T03:25:34.049 INFO:teuthology.nuke:Checking targets against current locks 2024-01-27T03:25:34.122 DEBUG:teuthology.nuke:shortname: smithi039 2024-01-27T03:25:34.122 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-27T03:25:34.147 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-26_01:08:12-rados-wip-yuri2-testing-2024-01-25-1327-reef-distro-default-smithi/7533659', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-27 03:05:54.607429', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDZwuH2VXyM0bW4EyhEdvnPK5w6AK2mMYCTmUpPqosW7wjIQY1NDCXB4dBAgWxOi3ozrrfCdNVhUzmqAWtxIv08='} 2024-01-27T03:25:34.151 DEBUG:teuthology.nuke:shortname: smithi112 2024-01-27T03:25:34.152 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-27T03:25:34.178 DEBUG:teuthology.nuke:shortname: smithi125 2024-01-27T03:25:34.178 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-27T03:25:34.184 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-26_01:08:12-rados-wip-yuri2-testing-2024-01-25-1327-reef-distro-default-smithi/7533659', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-27 03:05:54.608641', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJBkkfAFegdPMpcXKPUV0gl0vjgh4M1MWhae01ZV22KwehK4BxxjHpavN+bbYO9VLFdorXZVjC4T40KgUDDLrk8='} 2024-01-27T03:25:34.208 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-26_01:08:12-rados-wip-yuri2-testing-2024-01-25-1327-reef-distro-default-smithi/7533659', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-27 03:05:54.604962', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2024-01-27T03:25:34.221 INFO:teuthology.orchestra.console.smithi039:Power off 2024-01-27T03:25:34.221 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-27T03:25:34.236 DEBUG:teuthology.nuke:shortname: smithi151 2024-01-27T03:25:34.237 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-27T03:25:34.245 DEBUG:teuthology.orchestra.console.smithi039:power off output: Chassis Power Control: Down/Off 2024-01-27T03:25:34.245 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:34.256 DEBUG:teuthology.nuke:shortname: smithi176 2024-01-27T03:25:34.256 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-27T03:25:34.261 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-26_01:08:12-rados-wip-yuri2-testing-2024-01-25-1327-reef-distro-default-smithi/7533659', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-27 03:05:54.606096', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBbDWp6O4RBrC82oBzT2krv5Kc0N091SuRBYJ/zmQwIm2RWew8vnoC0Cmk5xg+afudjaY/XwWpwRTq0FT1UNA4E='} 2024-01-27T03:25:34.269 INFO:teuthology.orchestra.console.smithi112:Power off 2024-01-27T03:25:34.269 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-27T03:25:34.280 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi176.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-26_01:08:12-rados-wip-yuri2-testing-2024-01-25-1327-reef-distro-default-smithi/7533659', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-27 03:05:54.602968', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2024-01-27T03:25:34.285 INFO:teuthology.orchestra.console.smithi125:Power off 2024-01-27T03:25:34.286 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-27T03:25:34.298 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-01-27T03:25:34.301 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off 2024-01-27T03:25:34.301 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:34.313 DEBUG:teuthology.orchestra.console.smithi125:power off output: Chassis Power Control: Down/Off 2024-01-27T03:25:34.313 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:34.322 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-01-27T03:25:34.338 INFO:teuthology.orchestra.console.smithi151:Power off 2024-01-27T03:25:34.339 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-27T03:25:34.350 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2024-01-27T03:25:34.363 DEBUG:teuthology.orchestra.console.smithi151:power off output: Chassis Power Control: Down/Off 2024-01-27T03:25:34.363 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:34.379 INFO:teuthology.orchestra.console.smithi176:Power off 2024-01-27T03:25:34.379 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-27T03:25:34.388 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2024-01-27T03:25:34.403 DEBUG:teuthology.orchestra.console.smithi176:power off output: Chassis Power Control: Down/Off 2024-01-27T03:25:34.404 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:34.430 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is on 2024-01-27T03:25:38.302 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:38.324 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:38.351 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:38.389 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:38.413 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-01-27T03:25:38.431 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:38.440 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-01-27T03:25:38.462 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2024-01-27T03:25:38.500 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2024-01-27T03:25:38.540 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is on 2024-01-27T03:25:42.416 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:42.441 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:42.464 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:42.501 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:42.527 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-01-27T03:25:42.541 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:42.552 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-01-27T03:25:42.574 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2024-01-27T03:25:42.611 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2024-01-27T03:25:42.653 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is on 2024-01-27T03:25:46.528 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:46.554 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:46.575 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:46.612 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:46.638 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is off 2024-01-27T03:25:46.654 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T03:25:46.665 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-01-27T03:25:46.685 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off 2024-01-27T03:25:46.723 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is off 2024-01-27T03:25:46.740 INFO:teuthology.orchestra.console.smithi039:Power off completed 2024-01-27T03:25:46.765 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is off 2024-01-27T03:25:46.767 INFO:teuthology.orchestra.console.smithi112:Power off completed 2024-01-27T03:25:46.786 INFO:teuthology.orchestra.console.smithi125:Power off completed 2024-01-27T03:25:46.824 INFO:teuthology.orchestra.console.smithi151:Power off completed 2024-01-27T03:25:46.866 INFO:teuthology.orchestra.console.smithi176:Power off completed 2024-01-27T03:25:46.898 INFO:teuthology.lock.ops:unlocked: smithi039.front.sepia.ceph.com 2024-01-27T03:25:46.914 INFO:teuthology.lock.ops:unlocked: smithi112.front.sepia.ceph.com 2024-01-27T03:25:46.931 INFO:teuthology.lock.ops:unlocked: smithi125.front.sepia.ceph.com 2024-01-27T03:25:46.973 INFO:teuthology.lock.ops:unlocked: smithi151.front.sepia.ceph.com 2024-01-27T03:25:47.006 INFO:teuthology.lock.ops:unlocked: smithi176.front.sepia.ceph.com 2024-01-27T03:25:47.648 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/dispatcher/supervisor.py", line 58, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds