2023-09-02T01:26:57.190 INFO:root:teuthology version: 0.0.1.dev192+gf9a8556 2023-09-02T01:26:57.191 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-02T01:26:57.372 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc7d406adc0>, signals=[15]) 2023-09-02T01:26:57.374 INFO:teuthology.lock.ops:Start node 'smithi124.front.sepia.ceph.com' reimaging 2023-09-02T01:26:57.375 INFO:teuthology.lock.ops:Updating [smithi124.front.sepia.ceph.com]: reset os type and version on server 2023-09-02T01:26:57.375 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2023-09-02T01:26:57.396 INFO:teuthology.lock.ops:Start node 'smithi136.front.sepia.ceph.com' reimaging 2023-09-02T01:26:57.396 INFO:teuthology.lock.ops:Updating [smithi136.front.sepia.ceph.com]: reset os type and version on server 2023-09-02T01:26:57.397 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2023-09-02T01:26:57.399 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /host {"name": "smithi124"} 2023-09-02T01:26:57.421 INFO:teuthology.lock.ops:Start node 'smithi140.front.sepia.ceph.com' reimaging 2023-09-02T01:26:57.421 INFO:teuthology.lock.ops:Updating [smithi140.front.sepia.ceph.com]: reset os type and version on server 2023-09-02T01:26:57.422 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server 2023-09-02T01:26:57.423 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /host {"name": "smithi136"} 2023-09-02T01:26:57.446 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /host {"name": "smithi140"} 2023-09-02T01:26:57.470 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-09-02T01:26:57.497 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-09-02T01:26:57.512 DEBUG:teuthology.provision.fog.smithi124:Requesting image smithi_rhel_8.6 (ID 5) 2023-09-02T01:26:57.512 DEBUG:teuthology.provision.fog.smithi124:FOG request: PUT /host/229 {"imageID": 5} 2023-09-02T01:26:57.518 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-09-02T01:26:57.535 DEBUG:teuthology.provision.fog.smithi136:Requesting image smithi_rhel_8.6 (ID 5) 2023-09-02T01:26:57.536 DEBUG:teuthology.provision.fog.smithi136:FOG request: PUT /host/241 {"imageID": 5} 2023-09-02T01:26:57.556 DEBUG:teuthology.provision.fog.smithi140:Requesting image smithi_rhel_8.6 (ID 5) 2023-09-02T01:26:57.556 DEBUG:teuthology.provision.fog.smithi140:FOG request: PUT /host/245 {"imageID": 5} 2023-09-02T01:26:57.562 INFO:teuthology.provision.fog.smithi124:Scheduling deploy of rhel 8.6 2023-09-02T01:26:57.562 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-09-02T01:26:57.582 INFO:teuthology.provision.fog.smithi136:Scheduling deploy of rhel 8.6 2023-09-02T01:26:57.582 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-09-02T01:26:57.629 INFO:teuthology.provision.fog.smithi140:Scheduling deploy of rhel 8.6 2023-09-02T01:26:57.629 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-02T01:26:57.642 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /tasktype {"name": "deploy"} 2023-09-02T01:26:57.646 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /tasktype {"name": "deploy"} 2023-09-02T01:26:57.680 DEBUG:teuthology.provision.fog.smithi136:FOG request: POST /host/241/task {"taskTypeID": 1} 2023-09-02T01:26:57.681 DEBUG:teuthology.provision.fog.smithi124:FOG request: POST /host/229/task {"taskTypeID": 1} 2023-09-02T01:26:57.694 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /tasktype {"name": "deploy"} 2023-09-02T01:26:57.729 DEBUG:teuthology.provision.fog.smithi140:FOG request: POST /host/245/task {"taskTypeID": 1} 2023-09-02T01:26:57.732 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-09-02T01:26:57.735 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-09-02T01:26:57.780 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-02T01:26:57.844 INFO:teuthology.orchestra.console.smithi136:Power off 2023-09-02T01:26:57.846 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-02T01:26:57.867 INFO:teuthology.orchestra.console.smithi124:Power off 2023-09-02T01:26:57.868 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-02T01:26:57.880 DEBUG:teuthology.orchestra.console.smithi136:power off output: Chassis Power Control: Down/Off 2023-09-02T01:26:57.880 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:26:57.891 DEBUG:teuthology.orchestra.console.smithi124:power off output: Chassis Power Control: Down/Off 2023-09-02T01:26:57.891 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:26:57.906 INFO:teuthology.orchestra.console.smithi140:Power off 2023-09-02T01:26:57.907 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-02T01:26:57.914 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2023-09-02T01:26:57.915 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is off 2023-09-02T01:26:57.926 DEBUG:teuthology.orchestra.console.smithi140:power off output: Chassis Power Control: Down/Off 2023-09-02T01:26:57.927 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:26:57.948 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2023-09-02T01:26:58.016 INFO:teuthology.orchestra.console.smithi124:Power off completed 2023-09-02T01:26:58.016 INFO:teuthology.orchestra.console.smithi136:Power off completed 2023-09-02T01:26:58.049 INFO:teuthology.orchestra.console.smithi140:Power off completed 2023-09-02T01:26:58.117 INFO:teuthology.orchestra.console.smithi124:Power on 2023-09-02T01:26:58.117 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-02T01:26:58.125 INFO:teuthology.orchestra.console.smithi136:Power on 2023-09-02T01:26:58.125 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-02T01:26:58.139 DEBUG:teuthology.orchestra.console.smithi124:power on output: Chassis Power Control: Up/On 2023-09-02T01:26:58.140 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:26:58.148 DEBUG:teuthology.orchestra.console.smithi136:power on output: Chassis Power Control: Up/On 2023-09-02T01:26:58.149 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:26:58.156 INFO:teuthology.orchestra.console.smithi140:Power on 2023-09-02T01:26:58.157 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-02T01:26:58.166 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2023-09-02T01:26:58.169 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is off 2023-09-02T01:26:58.182 DEBUG:teuthology.orchestra.console.smithi140:power on output: Chassis Power Control: Up/On 2023-09-02T01:26:58.182 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:26:58.201 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2023-09-02T01:27:02.170 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:27:02.178 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:27:02.203 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:27:02.282 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on 2023-09-02T01:27:02.290 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is off 2023-09-02T01:27:02.316 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2023-09-02T01:27:02.386 INFO:teuthology.orchestra.console.smithi124:Power on completed 2023-09-02T01:27:02.487 INFO:teuthology.provision.fog.smithi124:Waiting for deploy to finish 2023-09-02T01:27:02.488 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-09-02T01:27:06.294 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:27:06.318 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:27:06.403 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2023-09-02T01:27:06.428 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2023-09-02T01:27:06.504 INFO:teuthology.orchestra.console.smithi136:Power on completed 2023-09-02T01:27:06.529 INFO:teuthology.orchestra.console.smithi140:Power on completed 2023-09-02T01:27:06.606 INFO:teuthology.provision.fog.smithi136:Waiting for deploy to finish 2023-09-02T01:27:06.606 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-09-02T01:27:06.630 INFO:teuthology.provision.fog.smithi140:Waiting for deploy to finish 2023-09-02T01:27:06.632 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-02T01:27:17.591 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-09-02T01:27:21.702 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-09-02T01:27:21.707 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-02T01:27:32.682 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-09-02T01:27:36.782 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-09-02T01:27:36.785 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-02T01:27:47.782 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-09-02T01:27:51.862 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-09-02T01:27:51.870 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-02T01:28:02.878 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-09-02T01:28:06.952 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-09-02T01:28:06.954 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-02T01:28:17.974 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-09-02T01:28:22.034 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-02T01:28:22.035 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-09-02T01:28:33.062 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-09-02T01:28:37.118 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-02T01:28:37.119 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-09-02T01:28:48.146 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-09-02T01:28:52.206 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-09-02T01:28:52.207 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-02T01:29:03.226 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-09-02T01:29:07.294 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-09-02T01:29:07.295 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-02T01:29:18.306 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-09-02T01:29:22.366 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-02T01:29:22.367 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-09-02T01:29:33.374 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-09-02T01:29:37.438 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-02T01:29:37.439 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-09-02T01:29:48.458 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-09-02T01:29:48.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:29:51.578 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-09-02T01:29:52.501 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-02T01:29:52.503 DEBUG:teuthology.provision.fog.smithi136:FOG request: GET /task/active None 2023-09-02T01:29:52.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:29:55.642 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-09-02T01:29:57.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:00.638 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-09-02T01:30:01.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:04.702 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-09-02T01:30:06.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:07.579 DEBUG:teuthology.provision.fog.smithi140:FOG request: GET /task/active None 2023-09-02T01:30:07.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:09.722 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-09-02T01:30:10.704 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:10.714 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:30:13.754 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-09-02T01:30:15.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:16.716 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:18.778 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-09-02T01:30:19.757 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:19.770 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:30:22.810 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-09-02T01:30:24.781 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:25.773 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:27.838 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-09-02T01:30:28.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:28.826 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:30:31.870 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-09-02T01:30:33.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:34.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:36.922 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-09-02T01:30:37.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:37.882 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:30:40.954 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-09-02T01:30:42.925 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:43.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:46.938 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:30:46.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:52.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:30:55.998 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:31:02.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:31:05.082 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:31:11.088 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:31:14.778 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-02T01:31:14.779 DEBUG:teuthology.orchestra.run.smithi124:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-02T01:31:15.070 INFO:teuthology.provision.fog.smithi124:Node is ready 2023-09-02T01:31:15.071 DEBUG:teuthology.orchestra.run.smithi124:> hostname 2023-09-02T01:31:15.088 INFO:teuthology.orchestra.run.smithi124.stdout:smithi022 2023-09-02T01:31:15.088 DEBUG:teuthology.orchestra.run.smithi124:> grep smithi022 /etc/hosts 2023-09-02T01:31:15.145 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022 2023-09-02T01:31:15.146 DEBUG:teuthology.orchestra.run.smithi124:> sudo hostname smithi124 2023-09-02T01:31:15.264 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i -e 's/smithi022/smithi124/g' /etc/hosts 2023-09-02T01:31:15.293 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i -e 's/172.21.15.22/172.21.15.124/g' /etc/hosts 2023-09-02T01:31:15.360 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i -e 's/smithi022/smithi124/g' /etc/hostname 2023-09-02T01:31:15.427 DEBUG:teuthology.orchestra.run.smithi124:> sudo hostname smithi124 2023-09-02T01:31:15.494 DEBUG:teuthology.orchestra.run.smithi124:> cat /etc/os-release 2023-09-02T01:31:15.551 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="Red Hat Enterprise Linux" 2023-09-02T01:31:15.551 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="8.6 (Ootpa)" 2023-09-02T01:31:15.551 INFO:teuthology.orchestra.run.smithi124.stdout:ID="rhel" 2023-09-02T01:31:15.552 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE="fedora" 2023-09-02T01:31:15.552 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="8.6" 2023-09-02T01:31:15.552 INFO:teuthology.orchestra.run.smithi124.stdout:PLATFORM_ID="platform:el8" 2023-09-02T01:31:15.552 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-02T01:31:15.553 INFO:teuthology.orchestra.run.smithi124.stdout:ANSI_COLOR="0;31" 2023-09-02T01:31:15.553 INFO:teuthology.orchestra.run.smithi124.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-02T01:31:15.553 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://www.redhat.com/" 2023-09-02T01:31:15.553 INFO:teuthology.orchestra.run.smithi124.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-02T01:31:15.553 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-02T01:31:15.554 INFO:teuthology.orchestra.run.smithi124.stdout: 2023-09-02T01:31:15.554 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-02T01:31:15.554 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-02T01:31:15.554 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-02T01:31:15.555 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-02T01:31:15.555 INFO:teuthology.provision.fog.smithi124:Deploy complete! 2023-09-02T01:31:15.558 DEBUG:teuthology.parallel:result is None 2023-09-02T01:31:46.988 WARNING:teuthology.provision.fog.smithi136:timed out 2023-09-02T01:31:52.994 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:31:53.350 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-02T01:31:53.351 DEBUG:teuthology.orchestra.run.smithi136:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-02T01:31:53.637 INFO:teuthology.provision.fog.smithi136:Node is ready 2023-09-02T01:31:53.637 DEBUG:teuthology.orchestra.run.smithi136:> hostname 2023-09-02T01:31:53.655 INFO:teuthology.orchestra.run.smithi136.stdout:smithi136.front.sepia.ceph.com 2023-09-02T01:31:53.655 DEBUG:teuthology.orchestra.run.smithi136:> grep smithi136.front.sepia.ceph.com /etc/hosts 2023-09-02T01:31:53.713 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.15.136 smithi136.front.sepia.ceph.com smithi136 2023-09-02T01:31:53.713 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname smithi136 2023-09-02T01:31:53.837 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i -e 's/smithi136.front.sepia.ceph.com/smithi136/g' /etc/hosts 2023-09-02T01:31:53.866 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i -e 's/172.21.15.136/172.21.15.136/g' /etc/hosts 2023-09-02T01:31:53.933 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i -e 's/smithi136.front.sepia.ceph.com/smithi136/g' /etc/hostname 2023-09-02T01:31:54.000 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname smithi136 2023-09-02T01:31:54.066 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2023-09-02T01:31:54.123 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="Red Hat Enterprise Linux" 2023-09-02T01:31:54.123 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="8.6 (Ootpa)" 2023-09-02T01:31:54.124 INFO:teuthology.orchestra.run.smithi136.stdout:ID="rhel" 2023-09-02T01:31:54.124 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="fedora" 2023-09-02T01:31:54.124 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="8.6" 2023-09-02T01:31:54.124 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el8" 2023-09-02T01:31:54.124 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-02T01:31:54.125 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2023-09-02T01:31:54.125 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-02T01:31:54.125 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://www.redhat.com/" 2023-09-02T01:31:54.125 INFO:teuthology.orchestra.run.smithi136.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-02T01:31:54.125 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-02T01:31:54.126 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-09-02T01:31:54.126 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-02T01:31:54.126 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-02T01:31:54.126 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-02T01:31:54.126 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-02T01:31:54.128 INFO:teuthology.provision.fog.smithi136:Deploy complete! 2023-09-02T01:31:54.128 DEBUG:teuthology.parallel:result is None 2023-09-02T01:32:11.106 WARNING:teuthology.provision.fog.smithi140:timed out 2023-09-02T01:32:17.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:32:20.190 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:32:26.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:32:29.278 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:32:35.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:32:38.362 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:32:44.366 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:32:47.418 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:32:53.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:32:56.506 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:33:02.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:33:05.594 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:33:11.601 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:33:14.682 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:33:20.688 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:33:23.738 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:33:29.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:33:32.794 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:33:38.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:33:41.854 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:33:47.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:33:50.938 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:33:56.942 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:33:59.994 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:34:05.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:34:09.054 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:34:15.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:34:18.142 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:34:24.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:34:27.226 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:34:33.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:34:36.314 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:34:42.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:34:45.370 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:34:51.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:34:54.426 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:35:00.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:35:03.486 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:35:09.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:35:12.570 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:35:18.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:35:21.626 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:35:27.633 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:35:30.718 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:35:36.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:35:39.802 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:35:45.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:35:48.858 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:35:54.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:35:57.918 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:36:03.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:36:07.006 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:36:13.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:36:16.090 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:36:22.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:36:25.150 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:36:31.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:36:34.238 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:36:40.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:36:43.322 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:36:49.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:36:52.378 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:36:58.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:37:01.438 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:37:07.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:37:10.522 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:37:16.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:37:19.610 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:37:25.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:37:28.670 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:37:34.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:37:37.754 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:37:43.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:37:46.842 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:37:52.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:37:55.902 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:38:01.907 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:38:04.990 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:38:10.994 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:38:14.078 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:38:20.085 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:38:23.162 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:38:29.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:38:32.218 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:38:38.222 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:38:41.278 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:38:47.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:38:50.362 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:38:56.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:38:59.450 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:39:05.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:39:08.506 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:39:14.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:39:17.562 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:39:23.566 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:39:26.618 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:39:32.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:39:35.674 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:39:41.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:39:44.730 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:39:50.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:39:53.790 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:39:59.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:40:02.878 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:40:08.882 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:40:11.962 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:40:17.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:40:21.022 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:40:27.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:40:30.106 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:40:36.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:40:39.166 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:40:45.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:40:48.254 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:40:54.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:40:57.338 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:41:03.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:41:06.402 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:41:12.406 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:41:15.482 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:41:21.486 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:41:24.538 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:41:30.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:41:33.598 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:41:39.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:41:42.682 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:41:48.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:41:51.742 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:41:57.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:42:00.826 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:42:06.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:42:09.914 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:42:15.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:42:18.970 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:42:24.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:42:28.026 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:42:34.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:42:37.114 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:42:43.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:42:46.174 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:42:52.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:42:55.258 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:43:01.265 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:43:04.350 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:43:10.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:43:13.438 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:43:19.443 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:43:22.522 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:43:28.526 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:43:31.578 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:43:37.582 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:43:40.638 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:43:46.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:43:49.722 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:43:55.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:43:58.778 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:44:04.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:44:07.870 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:44:13.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:44:16.958 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:44:22.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:44:26.042 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:44:32.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:44:35.130 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:44:41.134 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:44:44.190 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:44:50.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:44:53.274 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:44:59.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:45:02.330 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:45:08.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:45:11.390 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:45:17.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:45:20.442 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:45:26.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:45:29.498 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:45:35.502 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:45:38.554 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:45:44.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:45:47.610 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:45:53.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:45:56.670 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:46:02.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-02T01:46:05.754 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2023-09-02T01:46:05.756 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/fog.py", line 276, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2023-09-02T01:46:05.757 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-02T01:46:05.898 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=7af2ce9feef944f2b3580a6949044313 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/fog.py", line 276, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2023-09-02T01:46:05.901 INFO:teuthology.nuke:Checking targets against current locks 2023-09-02T01:46:05.966 DEBUG:teuthology.nuke:shortname: smithi124 2023-09-02T01:46:05.966 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-02T01:46:05.985 DEBUG:teuthology.nuke:shortname: smithi136 2023-09-02T01:46:05.985 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-02T01:46:05.988 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi124.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-02_01:04:01-fs-wip-batrick-testing-20230830.165602-distro-default-smithi/7386769', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-02 01:26:56.298477', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCo4YLO5ZulILuTTNnDRXDCoGOSPLU5T5Thvz20W83E+YIyncUziLZBhIO1tebUbxqPipwkTRUB4ahowQlPcJ3s8zavhnvv+d1vWGm3rqvc/DnYXUT4HFfye/59/OtV7XXOY+riEG3o7Rl8ZBagACLF98j+cEkzL+/XKKMEX6a4YFVV7rFTvwsfSTj86Wi2KVVFxjd01ZLEPbFMl/1DqRnWhW9CC1HQOjFtIB8BlzxmyWmMwi2QL2sUBUwP8RrJ/R3z88dwqTj8aBULkWiaFZBZ7Y1Z4rA1uX1ZYMV3oov0QEpfPFKA1szOSZyZn715hUyY2wcFHwRTb6on7ZcE6j3lH17DZn+tLYvH04U/pl2o6qeZYNmJWe9UknbQVUr+GLM38BIoieNUu1NK8C4121RFFWnjZRQlSFJmP9UalIrscIrtDqM73TN0pGZr0A9angzOdMrIxm5H3vMnD0/bBe37DntfX0WfdARLbuz4YX4ZE/HDfbd+gGzYQ/1yFpBoqis='} 2023-09-02T01:46:06.011 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-02_01:04:01-fs-wip-batrick-testing-20230830.165602-distro-default-smithi/7386769', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-02 01:26:56.300382', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJOSyJeiBL0MnTyQvbXoZ8lrC01nEKJyX/ZCZE2SLhS9qMKi9sEF7Ue4LgBK4/dZXj0rzbl/7HtkhIIifTyS7v4='} 2023-09-02T01:46:06.012 DEBUG:teuthology.nuke:shortname: smithi140 2023-09-02T01:46:06.013 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-02T01:46:06.038 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi140.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-09-02_01:04:01-fs-wip-batrick-testing-20230830.165602-distro-default-smithi/7386769', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-02 01:26:56.302062', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHp/IYoxO9/Gcls235TgBpWI/5Ut/3PSs6mGkz/E/BieQ97oQPTgEHnr3WqzCEA9mhukGkaQxFilDmGOYzSqZ4tWcGj+RJJASVVC+B/wNib4nWEV9Eu/awqVmc9BAEFlnIBl9tBzspC9Mcqrl/OUgE6A0qhoWljMSyvLagrFMY+RPnZM5SQBIHu+9X9FbIgOeCbCxEKIELNR5KES1A55NkmF6ScJUa8apDmhZVXcD6nTgVZQXraJU7WhUbY0XZq9XuZYvLr65Izm8XgS9p8Uktn0jcPbORfk4wNyGX/PsUImgZEMyu19a48Sk6suVw7UJd0nvrOp+MrnVyaVfXmdqr6HayJzyQE69gS0aH3F6jx/cph2KVXY8k0U5CZOVTQZE/dmIyOxzRU2/Mme1EnR98OPrzE81N3xYud8qIJ7EXqCbVpJ9chXK93KiFYHv165abJ34XLpBWXFyJTx3DKNEN7oEZj/OAKhqaBUYunqQu6UkblRryFNIzBPMGJZeLKBM='} 2023-09-02T01:46:06.044 INFO:teuthology.orchestra.console.smithi124:Power off 2023-09-02T01:46:06.045 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-02T01:46:06.066 DEBUG:teuthology.orchestra.console.smithi124:power off output: Chassis Power Control: Down/Off 2023-09-02T01:46:06.066 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:46:06.080 INFO:teuthology.orchestra.console.smithi136:Power off 2023-09-02T01:46:06.081 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-02T01:46:06.092 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on 2023-09-02T01:46:06.103 DEBUG:teuthology.orchestra.console.smithi136:power off output: Chassis Power Control: Down/Off 2023-09-02T01:46:06.103 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:46:06.120 INFO:teuthology.orchestra.console.smithi140:Power off 2023-09-02T01:46:06.121 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-02T01:46:06.129 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2023-09-02T01:46:06.144 DEBUG:teuthology.orchestra.console.smithi140:power off output: Chassis Power Control: Down/Off 2023-09-02T01:46:06.144 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:46:06.166 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2023-09-02T01:46:10.094 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:46:10.131 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:46:10.168 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:46:10.204 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on 2023-09-02T01:46:10.241 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2023-09-02T01:46:10.276 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2023-09-02T01:46:14.210 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:46:14.242 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:46:14.278 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-02T01:46:14.319 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2023-09-02T01:46:14.352 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is off 2023-09-02T01:46:14.387 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2023-09-02T01:46:14.421 INFO:teuthology.orchestra.console.smithi124:Power off completed 2023-09-02T01:46:14.453 INFO:teuthology.orchestra.console.smithi136:Power off completed 2023-09-02T01:46:14.489 INFO:teuthology.orchestra.console.smithi140:Power off completed 2023-09-02T01:46:14.573 INFO:teuthology.lock.ops:unlocked: smithi124.front.sepia.ceph.com 2023-09-02T01:46:14.606 INFO:teuthology.lock.ops:unlocked: smithi136.front.sepia.ceph.com 2023-09-02T01:46:14.646 INFO:teuthology.lock.ops:unlocked: smithi140.front.sepia.ceph.com 2023-09-02T01:46:15.055 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/dispatcher/supervisor.py", line 58, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/provision/fog.py", line 276, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds