2023-11-03T00:37:58.600 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-03T00:37:58.601 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-03T00:37:58.765 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7feb7596c670>, signals=[15]) 2023-11-03T00:37:58.767 INFO:teuthology.lock.ops:Start node 'smithi016.front.sepia.ceph.com' reimaging 2023-11-03T00:37:58.768 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: reset os type and version on server 2023-11-03T00:37:58.768 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-11-03T00:37:58.788 INFO:teuthology.lock.ops:Start node 'smithi140.front.sepia.ceph.com' reimaging 2023-11-03T00:37:58.788 INFO:teuthology.lock.ops:Updating [smithi140.front.sepia.ceph.com]: reset os type and version on server 2023-11-03T00:37:58.790 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server 2023-11-03T00:37:58.791 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /host {"name": "smithi016"} 2023-11-03T00:37:58.812 INFO:teuthology.lock.ops:Start node 'smithi204.front.sepia.ceph.com' reimaging 2023-11-03T00:37:58.813 INFO:teuthology.lock.ops:Updating [smithi204.front.sepia.ceph.com]: reset os type and version on server 2023-11-03T00:37:58.813 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2023-11-03T00:37:58.814 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /host {"name": "smithi140"} 2023-11-03T00:37:58.838 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /host {"name": "smithi204"} 2023-11-03T00:37:58.883 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-11-03T00:37:58.916 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-11-03T00:37:59.007 DEBUG:teuthology.provision.fog.smithi140:Requesting image smithi_rhel_8.6 (ID 5) 2023-11-03T00:37:59.007 DEBUG:teuthology.provision.fog.smithi140:FOG request: PUT /host/245 {"imageID": 5} 2023-11-03T00:37:59.009 DEBUG:teuthology.provision.fog.smithi204:Requesting image smithi_rhel_8.6 (ID 5) 2023-11-03T00:37:59.010 DEBUG:teuthology.provision.fog.smithi204:FOG request: PUT /host/308 {"imageID": 5} 2023-11-03T00:37:59.024 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-11-03T00:37:59.055 INFO:teuthology.provision.fog.smithi204:Scheduling deploy of rhel 8.6 2023-11-03T00:37:59.055 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-11-03T00:37:59.058 INFO:teuthology.provision.fog.smithi140:Scheduling deploy of rhel 8.6 2023-11-03T00:37:59.058 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-11-03T00:37:59.063 DEBUG:teuthology.provision.fog.smithi016:Requesting image smithi_rhel_8.6 (ID 5) 2023-11-03T00:37:59.064 DEBUG:teuthology.provision.fog.smithi016:FOG request: PUT /host/122 {"imageID": 5} 2023-11-03T00:37:59.098 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /tasktype {"name": "deploy"} 2023-11-03T00:37:59.104 INFO:teuthology.provision.fog.smithi016:Scheduling deploy of rhel 8.6 2023-11-03T00:37:59.104 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-11-03T00:37:59.107 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /tasktype {"name": "deploy"} 2023-11-03T00:37:59.134 DEBUG:teuthology.provision.fog.smithi204:FOG request: POST /host/308/task {"taskTypeID": 1} 2023-11-03T00:37:59.138 DEBUG:teuthology.provision.fog.smithi140:FOG request: POST /host/245/task {"taskTypeID": 1} 2023-11-03T00:37:59.146 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /tasktype {"name": "deploy"} 2023-11-03T00:37:59.179 DEBUG:teuthology.provision.fog.smithi016:FOG request: POST /host/122/task {"taskTypeID": 1} 2023-11-03T00:37:59.182 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-11-03T00:37:59.207 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-11-03T00:37:59.252 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-11-03T00:37:59.289 INFO:teuthology.orchestra.console.smithi204:Power off 2023-11-03T00:37:59.290 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-03T00:37:59.304 INFO:teuthology.orchestra.console.smithi140:Power off 2023-11-03T00:37:59.304 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-03T00:37:59.315 DEBUG:teuthology.orchestra.console.smithi204:power off output: Chassis Power Control: Down/Off 2023-11-03T00:37:59.316 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:37:59.324 DEBUG:teuthology.orchestra.console.smithi140:power off output: Chassis Power Control: Down/Off 2023-11-03T00:37:59.324 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:37:59.336 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2023-11-03T00:37:59.348 INFO:teuthology.orchestra.console.smithi016:Power off 2023-11-03T00:37:59.349 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-03T00:37:59.356 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2023-11-03T00:37:59.369 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2023-11-03T00:37:59.370 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:37:59.394 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-11-03T00:37:59.438 INFO:teuthology.orchestra.console.smithi204:Power off completed 2023-11-03T00:37:59.457 INFO:teuthology.orchestra.console.smithi140:Power off completed 2023-11-03T00:37:59.495 INFO:teuthology.orchestra.console.smithi016:Power off completed 2023-11-03T00:37:59.539 INFO:teuthology.orchestra.console.smithi204:Power on 2023-11-03T00:37:59.540 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-03T00:37:59.559 INFO:teuthology.orchestra.console.smithi140:Power on 2023-11-03T00:37:59.560 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-03T00:37:59.568 DEBUG:teuthology.orchestra.console.smithi204:power on output: Chassis Power Control: Up/On 2023-11-03T00:37:59.568 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:37:59.581 DEBUG:teuthology.orchestra.console.smithi140:power on output: Chassis Power Control: Up/On 2023-11-03T00:37:59.582 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:37:59.589 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2023-11-03T00:37:59.596 INFO:teuthology.orchestra.console.smithi016:Power on 2023-11-03T00:37:59.596 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-03T00:37:59.604 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2023-11-03T00:37:59.619 DEBUG:teuthology.orchestra.console.smithi016:power on output: Chassis Power Control: Up/On 2023-11-03T00:37:59.620 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:37:59.640 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-11-03T00:38:03.594 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:38:03.605 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:38:03.642 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:38:03.703 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2023-11-03T00:38:03.713 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2023-11-03T00:38:03.750 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-11-03T00:38:07.708 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:38:07.715 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:38:07.751 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:38:07.817 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2023-11-03T00:38:07.826 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2023-11-03T00:38:07.861 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2023-11-03T00:38:07.919 INFO:teuthology.orchestra.console.smithi204:Power on completed 2023-11-03T00:38:07.927 INFO:teuthology.orchestra.console.smithi140:Power on completed 2023-11-03T00:38:07.962 INFO:teuthology.orchestra.console.smithi016:Power on completed 2023-11-03T00:38:08.020 INFO:teuthology.provision.fog.smithi204:Waiting for deploy to finish 2023-11-03T00:38:08.021 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-11-03T00:38:08.028 INFO:teuthology.provision.fog.smithi140:Waiting for deploy to finish 2023-11-03T00:38:08.028 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-11-03T00:38:08.063 INFO:teuthology.provision.fog.smithi016:Waiting for deploy to finish 2023-11-03T00:38:08.064 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-11-03T00:38:23.086 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-11-03T00:38:23.087 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-11-03T00:38:23.112 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-11-03T00:38:38.152 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-11-03T00:38:38.154 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-11-03T00:38:38.162 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-11-03T00:38:53.222 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-11-03T00:38:53.224 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-11-03T00:38:53.225 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-11-03T00:39:08.304 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-11-03T00:39:08.305 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-11-03T00:39:08.306 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-11-03T00:39:23.378 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-11-03T00:39:23.379 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-11-03T00:39:23.380 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-11-03T00:39:38.462 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-11-03T00:39:38.463 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-11-03T00:39:38.464 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-11-03T00:39:53.535 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-11-03T00:39:53.538 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-11-03T00:39:53.541 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-11-03T00:40:08.610 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-11-03T00:40:08.611 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-11-03T00:40:08.612 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-11-03T00:40:23.690 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-11-03T00:40:23.691 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-11-03T00:40:23.692 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-11-03T00:40:38.770 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-11-03T00:40:38.771 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-11-03T00:40:38.772 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-11-03T00:40:38.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:40:41.914 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-11-03T00:40:47.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:40:50.970 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-11-03T00:40:53.846 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None 2023-11-03T00:40:53.851 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-11-03T00:40:53.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:40:56.954 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204 2023-11-03T00:40:56.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:00.026 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-11-03T00:41:02.958 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:06.010 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204 2023-11-03T00:41:06.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:08.913 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-11-03T00:41:08.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:09.082 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-11-03T00:41:12.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:12.058 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:41:15.066 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204 2023-11-03T00:41:15.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:18.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:18.138 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-11-03T00:41:21.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:21.118 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:41:24.122 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204 2023-11-03T00:41:24.139 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:27.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:27.194 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-11-03T00:41:30.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:30.202 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:41:33.185 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204 2023-11-03T00:41:33.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:36.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:39.189 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:39.258 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:41:42.266 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204 2023-11-03T00:41:45.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:48.270 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:48.314 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:41:51.322 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204 2023-11-03T00:41:54.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:57.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:41:57.370 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:42:03.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:42:06.430 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:42:12.434 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:42:33.218 WARNING:teuthology.provision.fog.smithi016:timed out 2023-11-03T00:42:39.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:42:39.563 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-03T00:42:39.564 DEBUG:teuthology.orchestra.run.smithi016:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-03T00:42:39.869 INFO:teuthology.provision.fog.smithi016:Node is ready 2023-11-03T00:42:39.870 DEBUG:teuthology.orchestra.run.smithi016:> hostname 2023-11-03T00:42:39.889 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016.front.sepia.ceph.com 2023-11-03T00:42:39.889 DEBUG:teuthology.orchestra.run.smithi016:> grep smithi016.front.sepia.ceph.com /etc/hosts 2023-11-03T00:42:39.949 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16 smithi016.front.sepia.ceph.com smithi016 2023-11-03T00:42:39.949 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname smithi016 2023-11-03T00:42:40.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/smithi016.front.sepia.ceph.com/smithi016/g' /etc/hosts 2023-11-03T00:42:40.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/172.21.15.16/172.21.15.16/g' /etc/hosts 2023-11-03T00:42:40.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/smithi016.front.sepia.ceph.com/smithi016/g' /etc/hostname 2023-11-03T00:42:40.239 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname smithi016 2023-11-03T00:42:40.309 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2023-11-03T00:42:40.367 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Red Hat Enterprise Linux" 2023-11-03T00:42:40.367 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8.6 (Ootpa)" 2023-11-03T00:42:40.368 INFO:teuthology.orchestra.run.smithi016.stdout:ID="rhel" 2023-11-03T00:42:40.368 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="fedora" 2023-11-03T00:42:40.368 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8.6" 2023-11-03T00:42:40.368 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2023-11-03T00:42:40.369 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-03T00:42:40.369 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2023-11-03T00:42:40.369 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-03T00:42:40.369 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.redhat.com/" 2023-11-03T00:42:40.369 INFO:teuthology.orchestra.run.smithi016.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-03T00:42:40.370 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-03T00:42:40.370 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-03T00:42:40.370 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-03T00:42:40.370 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-03T00:42:40.370 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-03T00:42:40.371 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-03T00:42:40.373 INFO:teuthology.provision.fog.smithi016:Deploy complete! 2023-11-03T00:42:40.374 DEBUG:teuthology.parallel:result is None 2023-11-03T00:42:57.342 WARNING:teuthology.provision.fog.smithi204:timed out 2023-11-03T00:43:03.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:43:03.685 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-03T00:43:03.686 DEBUG:teuthology.orchestra.run.smithi204:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-03T00:43:04.004 INFO:teuthology.provision.fog.smithi204:Node is ready 2023-11-03T00:43:04.004 DEBUG:teuthology.orchestra.run.smithi204:> hostname 2023-11-03T00:43:04.023 INFO:teuthology.orchestra.run.smithi204.stdout:smithi204.front.sepia.ceph.com 2023-11-03T00:43:04.023 DEBUG:teuthology.orchestra.run.smithi204:> grep smithi204.front.sepia.ceph.com /etc/hosts 2023-11-03T00:43:04.081 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.15.204 smithi204.front.sepia.ceph.com smithi204 2023-11-03T00:43:04.081 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname smithi204 2023-11-03T00:43:04.216 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i -e 's/smithi204.front.sepia.ceph.com/smithi204/g' /etc/hosts 2023-11-03T00:43:04.245 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i -e 's/172.21.15.204/172.21.15.204/g' /etc/hosts 2023-11-03T00:43:04.312 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i -e 's/smithi204.front.sepia.ceph.com/smithi204/g' /etc/hostname 2023-11-03T00:43:04.380 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname smithi204 2023-11-03T00:43:04.446 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2023-11-03T00:43:04.502 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="Red Hat Enterprise Linux" 2023-11-03T00:43:04.502 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="8.6 (Ootpa)" 2023-11-03T00:43:04.503 INFO:teuthology.orchestra.run.smithi204.stdout:ID="rhel" 2023-11-03T00:43:04.503 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="fedora" 2023-11-03T00:43:04.503 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="8.6" 2023-11-03T00:43:04.503 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el8" 2023-11-03T00:43:04.504 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-03T00:43:04.504 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2023-11-03T00:43:04.504 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-03T00:43:04.504 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://www.redhat.com/" 2023-11-03T00:43:04.504 INFO:teuthology.orchestra.run.smithi204.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-03T00:43:04.505 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-03T00:43:04.505 INFO:teuthology.orchestra.run.smithi204.stdout: 2023-11-03T00:43:04.505 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-03T00:43:04.505 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-03T00:43:04.505 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-03T00:43:04.506 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-03T00:43:04.506 INFO:teuthology.provision.fog.smithi204:Deploy complete! 2023-11-03T00:43:04.507 DEBUG:teuthology.parallel:result is None 2023-11-03T00:43:12.445 WARNING:teuthology.provision.fog.smithi140:timed out 2023-11-03T00:43:18.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:43:21.530 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:43:27.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:43:30.586 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:43:36.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:43:39.674 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:43:45.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:43:48.762 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:43:54.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:43:57.822 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:44:03.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:44:06.906 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:44:12.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:44:15.998 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:44:22.004 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:44:25.086 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:44:31.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:44:34.170 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:44:40.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:44:43.226 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:44:49.232 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:44:52.314 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:44:58.322 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:45:01.402 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:45:07.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:45:10.490 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:45:16.494 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:45:19.546 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:45:25.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:45:28.602 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:45:34.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:45:37.690 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:45:43.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:45:46.782 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:45:52.789 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:45:55.866 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:46:01.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:46:04.922 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:46:10.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:46:14.014 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:46:20.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:46:23.102 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:46:29.109 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:46:32.190 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:46:38.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:46:41.274 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:46:47.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:46:50.330 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:46:56.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:46:59.422 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:47:05.429 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:47:08.506 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:47:14.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:47:17.594 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:47:23.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:47:26.650 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:47:32.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:47:35.738 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:47:41.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:47:44.826 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:47:50.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:47:53.914 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:47:59.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:48:02.970 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:48:08.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:48:12.030 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:48:18.037 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:48:21.118 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:48:27.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:48:30.202 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:48:36.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:48:39.258 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:48:45.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:48:48.350 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:48:54.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:48:57.438 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:49:03.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:49:06.522 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:49:12.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:49:15.578 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:49:21.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:49:24.670 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:49:30.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:49:33.754 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:49:39.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:49:42.810 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:49:48.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:49:51.866 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:49:57.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:50:00.954 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:50:06.961 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:50:10.042 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:50:16.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:50:19.134 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:50:25.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:50:28.218 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:50:34.222 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:50:37.274 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:50:43.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:50:46.330 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:50:52.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:50:55.390 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:51:01.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:51:04.474 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:51:10.478 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:51:13.530 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:51:19.535 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:51:22.586 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:51:28.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:51:31.674 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:51:37.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:51:40.762 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:51:46.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:51:49.850 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:51:55.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:51:58.906 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:52:04.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:52:07.998 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:52:14.005 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:52:17.082 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:52:23.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:52:26.138 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:52:32.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:52:35.194 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:52:41.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:52:44.254 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:52:50.260 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:52:53.338 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:52:59.340 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:53:02.394 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:53:08.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:53:11.454 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:53:17.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:53:20.542 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:53:26.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:53:29.626 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:53:35.630 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:53:38.682 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:53:44.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:53:47.770 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:53:53.777 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:53:56.858 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:54:02.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:54:05.914 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:54:11.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:54:15.006 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:54:21.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:54:24.090 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:54:30.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:54:33.146 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:54:39.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:54:42.202 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:54:48.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:54:51.290 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:54:57.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:55:00.378 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:55:06.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:55:09.434 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:55:15.441 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:55:18.522 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:55:24.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:55:27.610 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:55:33.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:55:36.698 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:55:42.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:55:45.754 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:55:51.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:55:54.810 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:56:00.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:56:03.866 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:56:09.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:56:12.954 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:56:18.961 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:56:22.042 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:56:28.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:56:31.130 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:56:37.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:56:40.218 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:56:46.222 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:56:49.274 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:56:55.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:56:58.330 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:57:04.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-03T00:57:07.386 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-11-03T00:57:07.388 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 276, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2023-11-03T00:57:07.389 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-03T00:57:07.528 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=2a2a5e98754345ab8d62d3a52e443a80 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 276, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2023-11-03T00:57:07.530 INFO:teuthology.nuke:Checking targets against current locks 2023-11-03T00:57:07.598 DEBUG:teuthology.nuke:shortname: smithi016 2023-11-03T00:57:07.599 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-03T00:57:07.632 DEBUG:teuthology.nuke:shortname: smithi140 2023-11-03T00:57:07.632 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-03T00:57:07.633 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-01_14:51:50-rbd-wip-yuri3-testing-2023-10-31-1116-reef-distro-default-smithi/7443024', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-03 00:37:57.754799', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLJ3gnLRh32USQ5x6IawxiwvMS9ZoZTa3zyMpGi4S6VaXKmFuFYMXZjQFw1Wb985RjMI/Epo8NrgPqIvP95PKys='} 2023-11-03T00:57:07.657 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi140.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-01_14:51:50-rbd-wip-yuri3-testing-2023-10-31-1116-reef-distro-default-smithi/7443024', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-03 00:37:57.753787', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJRVCcU2vTqNAFa8EJEd5K9GRTBos47WJ/C9kF9apfzRdpBydsvNSxFfXjUBud+DyyQ4VUit27+bzKDg+U8dez0='} 2023-11-03T00:57:07.661 DEBUG:teuthology.nuke:shortname: smithi204 2023-11-03T00:57:07.662 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-03T00:57:07.694 INFO:teuthology.orchestra.console.smithi016:Power off 2023-11-03T00:57:07.695 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-03T00:57:07.704 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-01_14:51:50-rbd-wip-yuri3-testing-2023-10-31-1116-reef-distro-default-smithi/7443024', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-03 00:37:57.752386', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL96OlHfzjMIDvrzW/qbmV+FoaJpLnbg+cOzIGGQSg+ZVCf2CrXHML0DTvgvn4fRcUht4IRRa9qxvuHRtPNAKdQ='} 2023-11-03T00:57:07.720 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2023-11-03T00:57:07.721 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:57:07.735 INFO:teuthology.orchestra.console.smithi140:Power off 2023-11-03T00:57:07.735 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-03T00:57:07.748 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2023-11-03T00:57:07.757 DEBUG:teuthology.orchestra.console.smithi140:power off output: Chassis Power Control: Down/Off 2023-11-03T00:57:07.758 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:57:07.774 INFO:teuthology.orchestra.console.smithi204:Power off 2023-11-03T00:57:07.775 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-03T00:57:07.784 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2023-11-03T00:57:07.800 DEBUG:teuthology.orchestra.console.smithi204:power off output: Chassis Power Control: Down/Off 2023-11-03T00:57:07.800 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:57:07.823 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2023-11-03T00:57:11.753 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:57:11.786 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:57:11.825 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:57:11.865 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2023-11-03T00:57:11.896 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2023-11-03T00:57:11.934 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2023-11-03T00:57:15.870 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:57:15.897 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:57:15.934 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:57:15.980 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-11-03T00:57:16.006 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2023-11-03T00:57:16.044 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2023-11-03T00:57:16.082 INFO:teuthology.orchestra.console.smithi016:Power off completed 2023-11-03T00:57:16.235 INFO:teuthology.lock.ops:unlocked: smithi016.front.sepia.ceph.com 2023-11-03T00:57:20.010 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:57:20.045 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-03T00:57:20.120 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2023-11-03T00:57:20.155 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2023-11-03T00:57:20.222 INFO:teuthology.orchestra.console.smithi140:Power off completed 2023-11-03T00:57:20.257 INFO:teuthology.orchestra.console.smithi204:Power off completed 2023-11-03T00:57:20.378 INFO:teuthology.lock.ops:unlocked: smithi140.front.sepia.ceph.com 2023-11-03T00:57:20.402 INFO:teuthology.lock.ops:unlocked: smithi204.front.sepia.ceph.com 2023-11-03T00:57:20.789 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 58, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 276, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds