2023-09-03T11:14:22.439 INFO:root:teuthology version: 0.0.1.dev192+gf9a8556 2023-09-03T11:14:22.441 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-03T11:14:22.602 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbce8c57670>, signals=[15]) 2023-09-03T11:14:22.604 INFO:teuthology.lock.ops:Start node 'smithi057.front.sepia.ceph.com' reimaging 2023-09-03T11:14:22.604 INFO:teuthology.lock.ops:Updating [smithi057.front.sepia.ceph.com]: reset os type and version on server 2023-09-03T11:14:22.605 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server 2023-09-03T11:14:22.632 INFO:teuthology.lock.ops:Start node 'smithi140.front.sepia.ceph.com' reimaging 2023-09-03T11:14:22.633 INFO:teuthology.lock.ops:Updating [smithi140.front.sepia.ceph.com]: reset os type and version on server 2023-09-03T11:14:22.633 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server 2023-09-03T11:14:22.634 DEBUG:teuthology.provision.fog.smithi057:FOG request: GET /host {"name": "smithi057"} 2023-09-03T11:14:22.657 INFO:teuthology.lock.ops:Start node 'smithi155.front.sepia.ceph.com' reimaging 2023-09-03T11:14:22.657 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: reset os type and version on server 2023-09-03T11:14:22.658 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2023-09-03T11:14:22.659 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /host {"name": "smithi140"} 2023-09-03T11:14:22.674 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /host {"name": "smithi155"} 2023-09-03T11:14:22.717 DEBUG:teuthology.provision.fog.smithi057:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-09-03T11:14:22.742 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-09-03T11:14:22.745 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-09-03T11:14:22.756 DEBUG:teuthology.provision.fog.smithi057:Requesting image smithi_centos_9.stream (ID 10) 2023-09-03T11:14:22.757 DEBUG:teuthology.provision.fog.smithi057:FOG request: PUT /host/162 {"imageID": 10} 2023-09-03T11:14:22.784 DEBUG:teuthology.provision.fog.smithi155:Requesting image smithi_centos_9.stream (ID 10) 2023-09-03T11:14:22.784 DEBUG:teuthology.provision.fog.smithi155:FOG request: PUT /host/260 {"imageID": 10} 2023-09-03T11:14:22.789 DEBUG:teuthology.provision.fog.smithi140:Requesting image smithi_centos_9.stream (ID 10) 2023-09-03T11:14:22.790 DEBUG:teuthology.provision.fog.smithi140:FOG request: PUT /host/245 {"imageID": 10} 2023-09-03T11:14:22.801 INFO:teuthology.provision.fog.smithi057:Scheduling deploy of centos 9.stream 2023-09-03T11:14:22.801 DEBUG:teuthology.provision.fog.smithi057:FOG request: GET /task/active None 2023-09-03T11:14:22.823 INFO:teuthology.provision.fog.smithi155:Scheduling deploy of centos 9.stream 2023-09-03T11:14:22.824 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-03T11:14:22.832 INFO:teuthology.provision.fog.smithi140:Scheduling deploy of centos 9.stream 2023-09-03T11:14:22.832 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-03T11:14:22.859 DEBUG:teuthology.provision.fog.smithi057:FOG request: GET /tasktype {"name": "deploy"} 2023-09-03T11:14:22.868 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /tasktype {"name": "deploy"} 2023-09-03T11:14:22.880 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /tasktype {"name": "deploy"} 2023-09-03T11:14:22.896 DEBUG:teuthology.provision.fog.smithi057:FOG request: POST /host/162/task {"taskTypeID": 1} 2023-09-03T11:14:22.906 DEBUG:teuthology.provision.fog.smithi155:FOG request: POST /host/260/task {"taskTypeID": 1} 2023-09-03T11:14:22.910 DEBUG:teuthology.provision.fog.smithi140:FOG request: POST /host/245/task {"taskTypeID": 1} 2023-09-03T11:14:22.957 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-03T11:14:22.963 DEBUG:teuthology.provision.fog.smithi057:FOG request: GET /task/active None 2023-09-03T11:14:22.970 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-03T11:14:23.055 INFO:teuthology.orchestra.console.smithi155:Power off 2023-09-03T11:14:23.056 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-03T11:14:23.070 INFO:teuthology.orchestra.console.smithi140:Power off 2023-09-03T11:14:23.070 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-03T11:14:23.082 INFO:teuthology.orchestra.console.smithi057:Power off 2023-09-03T11:14:23.082 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-03T11:14:23.089 DEBUG:teuthology.orchestra.console.smithi155:power off output: Chassis Power Control: Down/Off 2023-09-03T11:14:23.090 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:14:23.097 DEBUG:teuthology.orchestra.console.smithi140:power off output: Chassis Power Control: Down/Off 2023-09-03T11:14:23.098 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:14:23.104 DEBUG:teuthology.orchestra.console.smithi057:power off output: Chassis Power Control: Down/Off 2023-09-03T11:14:23.105 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:14:23.113 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2023-09-03T11:14:23.116 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2023-09-03T11:14:23.128 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-09-03T11:14:23.214 INFO:teuthology.orchestra.console.smithi155:Power off completed 2023-09-03T11:14:23.218 INFO:teuthology.orchestra.console.smithi140:Power off completed 2023-09-03T11:14:23.316 INFO:teuthology.orchestra.console.smithi155:Power on 2023-09-03T11:14:23.316 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-03T11:14:23.324 INFO:teuthology.orchestra.console.smithi140:Power on 2023-09-03T11:14:23.324 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-03T11:14:23.337 DEBUG:teuthology.orchestra.console.smithi155:power on output: Chassis Power Control: Up/On 2023-09-03T11:14:23.338 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:14:23.346 DEBUG:teuthology.orchestra.console.smithi140:power on output: Chassis Power Control: Up/On 2023-09-03T11:14:23.347 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:14:23.357 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2023-09-03T11:14:23.368 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2023-09-03T11:14:27.133 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:14:27.242 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-09-03T11:14:27.359 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:14:27.369 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:14:27.469 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2023-09-03T11:14:27.478 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2023-09-03T11:14:27.571 INFO:teuthology.orchestra.console.smithi155:Power on completed 2023-09-03T11:14:27.580 INFO:teuthology.orchestra.console.smithi140:Power on completed 2023-09-03T11:14:27.672 INFO:teuthology.provision.fog.smithi155:Waiting for deploy to finish 2023-09-03T11:14:27.673 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-03T11:14:27.681 INFO:teuthology.provision.fog.smithi140:Waiting for deploy to finish 2023-09-03T11:14:27.681 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-03T11:14:31.246 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:14:31.354 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-09-03T11:14:35.359 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:14:35.469 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is off 2023-09-03T11:14:35.570 INFO:teuthology.orchestra.console.smithi057:Power off completed 2023-09-03T11:14:35.671 INFO:teuthology.orchestra.console.smithi057:Power on 2023-09-03T11:14:35.671 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-03T11:14:35.697 DEBUG:teuthology.orchestra.console.smithi057:power on output: Chassis Power Control: Up/On 2023-09-03T11:14:35.697 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:14:35.720 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is off 2023-09-03T11:14:39.722 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:14:39.834 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is off 2023-09-03T11:14:42.734 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-03T11:14:42.736 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-03T11:14:43.836 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:14:44.054 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-09-03T11:14:44.155 INFO:teuthology.orchestra.console.smithi057:Power on completed 2023-09-03T11:14:44.255 INFO:teuthology.provision.fog.smithi057:Waiting for deploy to finish 2023-09-03T11:14:44.256 DEBUG:teuthology.provision.fog.smithi057:FOG request: GET /task/active None 2023-09-03T11:14:57.806 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-03T11:14:57.807 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-03T11:14:59.317 DEBUG:teuthology.provision.fog.smithi057:FOG request: GET /task/active None 2023-09-03T11:15:12.878 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-03T11:15:12.879 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-03T11:15:14.378 DEBUG:teuthology.provision.fog.smithi057:FOG request: GET /task/active None 2023-09-03T11:15:27.954 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-03T11:15:27.955 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-03T11:15:29.436 DEBUG:teuthology.provision.fog.smithi057:FOG request: GET /task/active None 2023-09-03T11:15:43.026 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-03T11:15:43.027 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-03T11:15:44.492 DEBUG:teuthology.provision.fog.smithi057:FOG request: GET /task/active None 2023-09-03T11:15:58.090 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-03T11:15:58.092 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-03T11:15:59.545 DEBUG:teuthology.provision.fog.smithi057:FOG request: GET /task/active None 2023-09-03T11:16:13.154 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-03T11:16:13.155 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-03T11:16:14.605 DEBUG:teuthology.provision.fog.smithi057:FOG request: GET /task/active None 2023-09-03T11:16:28.230 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-03T11:16:28.232 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-03T11:16:29.662 DEBUG:teuthology.provision.fog.smithi057:FOG request: GET /task/active None 2023-09-03T11:16:43.298 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-03T11:16:43.301 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-03T11:16:44.737 DEBUG:teuthology.provision.fog.smithi057:FOG request: GET /task/active None 2023-09-03T11:16:58.376 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-03T11:16:58.378 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-03T11:16:59.802 DEBUG:teuthology.provision.fog.smithi057:FOG request: GET /task/active None 2023-09-03T11:17:13.450 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-03T11:17:13.451 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-03T11:17:13.505 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:17:13.508 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:17:14.858 DEBUG:teuthology.provision.fog.smithi057:FOG request: GET /task/active None 2023-09-03T11:17:14.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:17:15.035 ERROR:teuthology.orchestra.connection:Error authenticating with smithi057.front.sepia.ceph.com: Authentication failed. 2023-09-03T11:17:15.037 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-03T11:17:15.038 DEBUG:teuthology.orchestra.run.smithi057:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-03T11:17:15.039 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-03T11:17:16.045 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-03T11:17:17.051 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-03T11:17:18.058 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-03T11:17:31.934 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-09-03T11:17:31.935 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:17:37.936 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:17:37.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:17:41.018 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-09-03T11:17:41.019 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:17:47.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:17:47.021 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:17:50.074 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-09-03T11:17:50.075 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:17:56.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:17:56.078 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:17:59.130 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-09-03T11:17:59.130 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:18:05.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:18:05.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:18:08.190 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-09-03T11:18:08.190 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:18:14.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:18:14.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:19:06.394 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-09-03T11:19:06.396 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi057 2023-09-03T11:19:14.201 WARNING:teuthology.provision.fog.smithi155:timed out 2023-09-03T11:19:14.202 WARNING:teuthology.provision.fog.smithi140:timed out 2023-09-03T11:19:15.398 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-09-03T11:19:15.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:19:15.631 DEBUG:teuthology.orchestra.run.smithi057:> true 2023-09-03T11:19:15.959 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-09-03T11:19:15.960 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-03T11:19:15.960 DEBUG:teuthology.orchestra.run.smithi057:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-03T11:19:20.204 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:19:20.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:19:20.445 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-03T11:19:20.446 DEBUG:teuthology.orchestra.run.smithi155:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-03T11:19:20.782 INFO:teuthology.provision.fog.smithi155:Node is ready 2023-09-03T11:19:20.782 DEBUG:teuthology.orchestra.run.smithi155:> hostname 2023-09-03T11:19:20.797 INFO:teuthology.orchestra.run.smithi155.stdout:smithi155.front.sepia.ceph.com 2023-09-03T11:19:20.797 DEBUG:teuthology.orchestra.run.smithi155:> grep smithi155.front.sepia.ceph.com /etc/hosts 2023-09-03T11:19:20.852 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.15.155 smithi155.front.sepia.ceph.com smithi155 2023-09-03T11:19:20.852 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname smithi155 2023-09-03T11:19:20.970 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i -e 's/smithi155.front.sepia.ceph.com/smithi155/g' /etc/hosts 2023-09-03T11:19:20.975 INFO:teuthology.provision.fog.smithi057:Node is ready 2023-09-03T11:19:20.976 DEBUG:teuthology.orchestra.run.smithi057:> hostname 2023-09-03T11:19:20.994 INFO:teuthology.orchestra.run.smithi057.stdout:smithi046 2023-09-03T11:19:20.994 DEBUG:teuthology.orchestra.run.smithi057:> grep smithi046 /etc/hosts 2023-09-03T11:19:20.998 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i -e 's/172.21.15.155/172.21.15.155/g' /etc/hosts 2023-09-03T11:19:21.051 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2023-09-03T11:19:21.052 DEBUG:teuthology.orchestra.run.smithi057:> sudo hostname smithi057 2023-09-03T11:19:21.065 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i -e 's/smithi155.front.sepia.ceph.com/smithi155/g' /etc/hostname 2023-09-03T11:19:21.132 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname smithi155 2023-09-03T11:19:21.171 DEBUG:teuthology.orchestra.run.smithi057:> sudo sed -i -e 's/smithi046/smithi057/g' /etc/hosts 2023-09-03T11:19:21.198 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2023-09-03T11:19:21.206 DEBUG:teuthology.orchestra.run.smithi057:> sudo sed -i -e 's/172.21.15.46/172.21.15.57/g' /etc/hosts 2023-09-03T11:19:21.253 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Stream" 2023-09-03T11:19:21.254 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="9" 2023-09-03T11:19:21.254 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2023-09-03T11:19:21.254 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2023-09-03T11:19:21.254 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="9" 2023-09-03T11:19:21.255 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el9" 2023-09-03T11:19:21.255 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Stream 9" 2023-09-03T11:19:21.255 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2023-09-03T11:19:21.255 INFO:teuthology.orchestra.run.smithi155.stdout:LOGO="fedora-logo-icon" 2023-09-03T11:19:21.255 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-09-03T11:19:21.256 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/" 2023-09-03T11:19:21.256 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-03T11:19:21.256 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-09-03T11:19:21.256 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-03T11:19:21.258 INFO:teuthology.provision.fog.smithi155:Deploy complete! 2023-09-03T11:19:21.260 DEBUG:teuthology.parallel:result is None 2023-09-03T11:19:21.280 DEBUG:teuthology.orchestra.run.smithi057:> sudo sed -i -e 's/smithi046/smithi057/g' /etc/hostname 2023-09-03T11:19:21.354 DEBUG:teuthology.orchestra.run.smithi057:> sudo hostname smithi057 2023-09-03T11:19:21.425 DEBUG:teuthology.orchestra.run.smithi057:> cat /etc/os-release 2023-09-03T11:19:21.482 INFO:teuthology.orchestra.run.smithi057.stdout:NAME="CentOS Stream" 2023-09-03T11:19:21.482 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION="9" 2023-09-03T11:19:21.482 INFO:teuthology.orchestra.run.smithi057.stdout:ID="centos" 2023-09-03T11:19:21.483 INFO:teuthology.orchestra.run.smithi057.stdout:ID_LIKE="rhel fedora" 2023-09-03T11:19:21.483 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_ID="9" 2023-09-03T11:19:21.483 INFO:teuthology.orchestra.run.smithi057.stdout:PLATFORM_ID="platform:el9" 2023-09-03T11:19:21.483 INFO:teuthology.orchestra.run.smithi057.stdout:PRETTY_NAME="CentOS Stream 9" 2023-09-03T11:19:21.484 INFO:teuthology.orchestra.run.smithi057.stdout:ANSI_COLOR="0;31" 2023-09-03T11:19:21.484 INFO:teuthology.orchestra.run.smithi057.stdout:LOGO="fedora-logo-icon" 2023-09-03T11:19:21.484 INFO:teuthology.orchestra.run.smithi057.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-09-03T11:19:21.484 INFO:teuthology.orchestra.run.smithi057.stdout:HOME_URL="https://centos.org/" 2023-09-03T11:19:21.485 INFO:teuthology.orchestra.run.smithi057.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-03T11:19:21.485 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-09-03T11:19:21.485 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-03T11:19:21.485 INFO:teuthology.provision.fog.smithi057:Deploy complete! 2023-09-03T11:19:21.486 DEBUG:teuthology.parallel:result is None 2023-09-03T11:19:38.650 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:19:44.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:19:47.738 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:19:53.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:19:56.794 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:20:02.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:20:05.850 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:20:11.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:20:14.910 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:20:20.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:20:23.998 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:20:30.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:20:33.086 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:20:39.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:20:42.174 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:20:48.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:20:51.265 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:20:57.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:21:00.348 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:21:06.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:21:09.434 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:21:15.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:21:18.494 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:21:24.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:21:27.578 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:21:33.582 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:21:36.634 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:21:42.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:21:45.694 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:21:51.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:21:54.782 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:22:00.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:22:03.866 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:22:09.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:22:12.922 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:22:18.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:22:22.010 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:22:28.017 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:22:31.098 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:22:37.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:22:40.186 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:22:46.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:22:49.278 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:22:55.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:22:58.362 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:23:04.366 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:23:07.418 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:23:13.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:23:16.506 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:23:22.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:23:25.562 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:23:31.569 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:23:34.654 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:23:40.660 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:23:43.738 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:23:49.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:23:52.826 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:23:58.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:24:01.882 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:24:07.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:24:10.938 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:24:16.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:24:19.994 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:24:26.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:24:29.082 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:24:35.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:24:38.138 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:24:44.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:24:47.226 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:24:53.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:24:56.282 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:25:02.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:25:05.338 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:25:11.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:25:14.426 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:25:20.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:25:23.482 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:25:29.487 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:25:32.538 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:25:38.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:25:41.594 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:25:47.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:25:50.654 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:25:56.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:25:59.742 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:26:05.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:26:08.830 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:26:14.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:26:17.914 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:26:23.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:26:26.974 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:26:32.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:26:36.058 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:26:42.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:26:45.118 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:26:51.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:26:54.202 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:27:00.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:27:03.258 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:27:09.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:27:12.314 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:27:18.316 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:27:21.374 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:27:27.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:27:30.462 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:27:36.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:27:39.546 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:27:45.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:27:48.634 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:27:54.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:27:57.726 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:28:03.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:28:06.814 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:28:12.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:28:15.898 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:28:21.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:28:24.954 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:28:30.961 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:28:34.042 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:28:40.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:28:43.134 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:28:49.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:28:52.218 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:28:58.222 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:29:01.274 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:29:07.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:29:10.330 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:29:16.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:29:19.386 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:29:25.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:29:28.446 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:29:34.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:29:37.530 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:29:43.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:29:46.586 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:29:52.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:29:55.642 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:30:01.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:30:04.698 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:30:10.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:30:13.759 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:30:19.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:30:22.842 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:30:28.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:30:31.898 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:30:37.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:30:40.954 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:30:46.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:30:50.046 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:30:56.053 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:30:59.130 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:31:05.134 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:31:08.186 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:31:14.190 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:31:17.242 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:31:23.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:31:26.301 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:31:32.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:31:35.390 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:31:41.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:31:44.478 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:31:50.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:31:53.566 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:31:59.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:32:02.650 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:32:08.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:32:11.706 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:32:17.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:32:20.794 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:32:26.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:32:29.882 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:32:35.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:32:38.970 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:32:44.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:32:48.026 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:32:54.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:32:57.118 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:33:03.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:33:06.202 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:33:12.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:33:15.290 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:33:21.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:33:24.346 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:33:30.350 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:33:33.402 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:33:39.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-03T11:33:42.458 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-03T11:33:42.460 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/fog.py", line 276, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2023-09-03T11:33:42.461 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-03T11:33:42.607 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=bbd48c847d2d422080bc45124267c925 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/fog.py", line 276, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2023-09-03T11:33:42.610 INFO:teuthology.nuke:Checking targets against current locks 2023-09-03T11:33:42.681 DEBUG:teuthology.nuke:shortname: smithi057 2023-09-03T11:33:42.681 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-03T11:33:42.704 DEBUG:teuthology.nuke:shortname: smithi140 2023-09-03T11:33:42.705 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-03T11:33:42.708 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2023-09-03_10:29:06-rbd-wip-dis-testing-distro-default-smithi/7387588', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-03 11:14:21.621443', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0='} 2023-09-03T11:33:42.727 DEBUG:teuthology.nuke:shortname: smithi155 2023-09-03T11:33:42.727 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-03T11:33:42.733 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi140.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2023-09-03_10:29:06-rbd-wip-dis-testing-distro-default-smithi/7387588', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-03 11:14:21.617756', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOX8jFPx0HAdkH/SLi/pJmfC3McdT26JGaPHhwdKvD6c'} 2023-09-03T11:33:42.750 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2023-09-03_10:29:06-rbd-wip-dis-testing-distro-default-smithi/7387588', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-03 11:14:21.619714', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIP58k6WKOBqj07h8qyYnUGsFd5XdSF0R1c7lsfW4abYF'} 2023-09-03T11:33:42.760 INFO:teuthology.orchestra.console.smithi057:Power off 2023-09-03T11:33:42.761 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-03T11:33:42.787 DEBUG:teuthology.orchestra.console.smithi057:power off output: Chassis Power Control: Down/Off 2023-09-03T11:33:42.788 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:33:42.808 INFO:teuthology.orchestra.console.smithi140:Power off 2023-09-03T11:33:42.809 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-03T11:33:42.818 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-09-03T11:33:42.830 DEBUG:teuthology.orchestra.console.smithi140:power off output: Chassis Power Control: Down/Off 2023-09-03T11:33:42.830 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:33:42.845 INFO:teuthology.orchestra.console.smithi155:Power off 2023-09-03T11:33:42.845 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-03T11:33:42.854 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2023-09-03T11:33:42.869 DEBUG:teuthology.orchestra.console.smithi155:power off output: Chassis Power Control: Down/Off 2023-09-03T11:33:42.869 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:33:42.901 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2023-09-03T11:33:46.823 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:33:46.855 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:33:46.902 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:33:46.934 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-09-03T11:33:46.966 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2023-09-03T11:33:47.012 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2023-09-03T11:33:50.938 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:33:50.968 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:33:51.013 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:33:51.048 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-09-03T11:33:51.079 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2023-09-03T11:33:51.124 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2023-09-03T11:33:51.225 INFO:teuthology.orchestra.console.smithi155:Power off completed 2023-09-03T11:33:51.370 INFO:teuthology.lock.ops:unlocked: smithi155.front.sepia.ceph.com 2023-09-03T11:33:55.054 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:33:55.080 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-03T11:33:55.163 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is off 2023-09-03T11:33:55.190 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2023-09-03T11:33:55.264 INFO:teuthology.orchestra.console.smithi057:Power off completed 2023-09-03T11:33:55.292 INFO:teuthology.orchestra.console.smithi140:Power off completed 2023-09-03T11:33:55.419 INFO:teuthology.lock.ops:unlocked: smithi057.front.sepia.ceph.com 2023-09-03T11:33:55.452 INFO:teuthology.lock.ops:unlocked: smithi140.front.sepia.ceph.com 2023-09-03T11:33:55.785 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/dispatcher/supervisor.py", line 58, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/fog.py", line 276, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds