2023-06-29T21:14:41.437 INFO:root:teuthology version: 0.0.1.dev134+ge25bf39 2023-06-29T21:14:41.439 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-29T21:14:41.439 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-29T21:14:41.561 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f05470e2550>, signals=[15]) 2023-06-29T21:14:41.562 INFO:teuthology.lock.ops:Start node 'smithi027.front.sepia.ceph.com' reimaging 2023-06-29T21:14:41.562 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: reset os type and version on server 2023-06-29T21:14:41.563 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2023-06-29T21:14:41.589 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /host {"name": "smithi027"} 2023-06-29T21:14:41.672 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /image {"name": "smithi_ubuntu_22.04"} 2023-06-29T21:14:41.716 DEBUG:teuthology.provision.fog.smithi027:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-06-29T21:14:41.717 DEBUG:teuthology.provision.fog.smithi027:FOG request: PUT /host/132 {"imageID": 6} 2023-06-29T21:14:41.759 INFO:teuthology.provision.fog.smithi027:Scheduling deploy of ubuntu 22.04 2023-06-29T21:14:41.759 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:14:41.818 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /tasktype {"name": "deploy"} 2023-06-29T21:14:41.856 DEBUG:teuthology.provision.fog.smithi027:FOG request: POST /host/132/task {"taskTypeID": 1} 2023-06-29T21:14:41.913 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:14:42.021 INFO:teuthology.orchestra.console:Power off smithi027 2023-06-29T21:14:42.022 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-29T21:14:42.048 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-29T21:14:42.174 INFO:teuthology.orchestra.console:Power off for smithi027 completed 2023-06-29T21:14:42.275 INFO:teuthology.orchestra.console:Power on smithi027 2023-06-29T21:14:42.276 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-06-29T21:14:42.300 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-29T21:14:46.328 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-29T21:14:50.440 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-29T21:14:50.655 INFO:teuthology.orchestra.console:Power on for smithi027 completed 2023-06-29T21:14:50.757 INFO:teuthology.provision.fog.smithi027:Waiting for deploy to finish 2023-06-29T21:14:50.757 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:15:05.842 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:15:20.938 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:15:36.033 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:15:51.120 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:16:06.212 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:16:21.310 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:16:36.380 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:16:51.448 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:17:06.525 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:17:21.601 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:17:36.676 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:17:51.752 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:18:06.818 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:18:21.896 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:18:36.971 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:18:52.060 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:19:07.153 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:19:22.237 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:19:37.328 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:19:52.410 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:20:07.498 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:20:22.584 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:20:37.660 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:20:52.740 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:21:07.825 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:21:22.900 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:21:37.971 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:21:53.044 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:22:08.107 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:22:23.177 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:22:38.252 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:22:53.315 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:23:08.391 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:23:23.465 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:23:38.526 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:23:53.599 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:24:08.673 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:24:23.747 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:24:38.816 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:24:53.897 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:25:08.967 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:25:24.051 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:25:39.112 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:25:54.187 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:26:09.263 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:26:24.333 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:26:39.412 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:26:54.491 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:27:09.568 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:27:24.648 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:27:39.726 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:27:54.807 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:28:09.891 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:28:24.961 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:28:40.041 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:28:55.122 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:29:10.198 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:29:25.287 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:29:40.352 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:29:55.425 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:30:10.504 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:30:25.585 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:30:40.661 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:30:55.737 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:31:10.808 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:31:25.887 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:31:40.968 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:31:56.047 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:32:11.122 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:32:26.198 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:32:41.270 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:32:56.351 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:33:11.419 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:33:26.489 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:33:41.558 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:33:56.627 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:34:11.704 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:34:26.780 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:34:41.858 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:34:56.939 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:35:12.016 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:35:27.096 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:35:42.172 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:35:57.236 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:36:12.316 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:36:27.394 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:36:42.458 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:36:57.531 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:37:12.603 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:37:27.676 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:37:42.748 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:37:57.835 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:38:12.898 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:38:27.969 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:38:43.044 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:38:58.113 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:39:13.195 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:39:28.259 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:39:43.332 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:39:58.399 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:40:13.477 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:40:28.565 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:40:43.636 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:40:58.712 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:41:13.790 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:41:28.868 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:41:43.937 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:41:59.022 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:42:14.100 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:42:29.174 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:42:44.249 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:42:59.324 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:43:14.393 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:43:29.479 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:43:44.559 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:43:59.620 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:44:14.700 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:44:29.765 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:44:44.836 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:44:59.912 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:45:15.000 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:45:30.077 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:45:45.139 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:46:00.215 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:46:15.294 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:46:30.376 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:46:45.454 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:47:00.534 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:47:15.612 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:47:30.693 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:47:45.776 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:48:00.857 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:48:15.933 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:48:31.005 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:48:46.065 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:49:01.132 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:49:16.198 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:49:31.276 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:49:46.350 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:50:01.417 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:50:16.490 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:50:31.570 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:50:46.635 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:51:01.697 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:51:16.771 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:51:31.844 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:51:46.910 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:52:01.985 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:52:17.060 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:52:32.131 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:52:47.211 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:53:02.286 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:53:17.362 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:53:32.438 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:53:47.500 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:54:02.575 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:54:17.649 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:54:32.733 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:54:47.812 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:55:02.889 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:55:17.966 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:55:33.039 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:55:48.116 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:56:03.183 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:56:18.256 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:56:33.323 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:56:48.401 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:57:03.481 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:57:18.551 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:57:33.636 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:57:48.708 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:58:03.787 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:58:18.864 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:58:33.941 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:58:49.014 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:59:04.081 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:59:19.152 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:59:34.232 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T21:59:49.308 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:00:04.384 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:00:19.461 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:00:34.535 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:00:49.609 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:01:04.682 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:01:19.758 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:01:34.840 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:01:49.916 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:02:04.993 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:02:20.068 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:02:35.147 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:02:50.229 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:03:05.313 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:03:20.381 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:03:35.448 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:03:50.518 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:04:05.589 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:04:20.668 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:04:35.748 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:04:50.828 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:05:05.906 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:05:20.978 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:05:36.046 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:05:51.118 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:06:06.188 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:06:21.267 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:06:36.340 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:06:51.404 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:07:06.467 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:07:21.528 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:07:36.608 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:07:51.681 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:08:06.760 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:08:21.848 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:08:36.929 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:08:52.008 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:09:07.092 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:09:22.174 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:09:37.240 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:09:52.329 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:10:07.410 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:10:22.499 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:10:37.580 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:10:52.662 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:11:07.739 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:11:22.819 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:11:37.906 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:11:52.987 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:12:08.047 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:12:23.115 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:12:38.193 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:12:53.260 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:13:08.331 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:13:23.406 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:13:38.482 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:13:53.553 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:14:08.632 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:14:23.697 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:14:38.761 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:14:53.833 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-06-29T22:14:53.899 DEBUG:teuthology.provision.fog.smithi027:Canceling deploy task with ID 355418 2023-06-29T22:14:53.899 DEBUG:teuthology.provision.fog.smithi027:FOG request: DELETE /task/cancel {"id": 355418} 2023-06-29T22:14:53.945 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/provision/fog.py", line 86, in create self.wait_for_deploy_task(task_id) File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/provision/fog.py", line 259, in wait_for_deploy_task while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (241) after waiting for 3600 seconds 2023-06-29T22:14:53.945 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-29T22:14:53.946 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-29T22:14:54.022 INFO:teuthology.nuke:Checking targets against current locks 2023-06-29T22:14:54.043 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/provision/fog.py", line 86, in create self.wait_for_deploy_task(task_id) File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/provision/fog.py", line 259, in wait_for_deploy_task while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (241) after waiting for 3600 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/dispatcher/supervisor.py", line 57, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/dispatcher/supervisor.py", line 235, in reimage nuke.nuke(ctx, True) File "/home/teuthworker/src/git.ceph.com_teuthology_e25bf390964c19fabac6a4bf8efd783aaa7f157c/teuthology/nuke/__init__.py", line 248, in nuke if ctx.name and ctx.name not in status.get('description', ""): TypeError: argument of type 'NoneType' is not iterable