2023-08-10T10:20:48.755 INFO:root:teuthology version: 0.0.1.dev181+g7fda959
2023-08-10T10:20:48.757 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-10T10:20:48.890 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fd9500b9820>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fd94f362ee0>, signals=[15])
2023-08-10T10:20:48.892 INFO:teuthology.lock.ops:Start node 'smithi074.front.sepia.ceph.com' reimaging
2023-08-10T10:20:48.892 INFO:teuthology.lock.ops:Updating [smithi074.front.sepia.ceph.com]: reset os type and version on server
2023-08-10T10:20:48.893 INFO:teuthology.lock.ops:Updating smithi074.front.sepia.ceph.com on lock server
2023-08-10T10:20:48.912 INFO:teuthology.lock.ops:Start node 'smithi179.front.sepia.ceph.com' reimaging
2023-08-10T10:20:48.913 INFO:teuthology.lock.ops:Updating [smithi179.front.sepia.ceph.com]: reset os type and version on server
2023-08-10T10:20:48.913 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server
2023-08-10T10:20:48.915 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /host {"name": "smithi074"}
2023-08-10T10:20:48.937 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /host {"name": "smithi179"}
2023-08-10T10:20:48.996 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-08-10T10:20:49.004 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-08-10T10:20:49.035 DEBUG:teuthology.provision.fog.smithi074:Requesting image smithi_centos_8.stream (ID 3)
2023-08-10T10:20:49.035 DEBUG:teuthology.provision.fog.smithi074:FOG request: PUT /host/179 {"imageID": 3}
2023-08-10T10:20:49.045 DEBUG:teuthology.provision.fog.smithi179:Requesting image smithi_centos_8.stream (ID 3)
2023-08-10T10:20:49.045 DEBUG:teuthology.provision.fog.smithi179:FOG request: PUT /host/284 {"imageID": 3}
2023-08-10T10:20:49.073 INFO:teuthology.provision.fog.smithi074:Scheduling deploy of centos 8.stream
2023-08-10T10:20:49.074 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:20:49.093 INFO:teuthology.provision.fog.smithi179:Scheduling deploy of centos 8.stream
2023-08-10T10:20:49.094 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None
2023-08-10T10:20:49.122 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /tasktype {"name": "deploy"}
2023-08-10T10:20:49.142 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /tasktype {"name": "deploy"}
2023-08-10T10:20:49.154 DEBUG:teuthology.provision.fog.smithi074:FOG request: POST /host/179/task {"taskTypeID": 1}
2023-08-10T10:20:49.187 DEBUG:teuthology.provision.fog.smithi179:FOG request: POST /host/284/task {"taskTypeID": 1}
2023-08-10T10:20:49.206 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:20:49.243 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None
2023-08-10T10:20:49.280 INFO:teuthology.orchestra.console.smithi074:Power off
2023-08-10T10:20:49.281 DEBUG:teuthology.orchestra.console.smithi074:pexpect command: ipmitool -H smithi074.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-10T10:20:49.303 DEBUG:teuthology.orchestra.console.smithi074:power off output: Chassis Power Control: Down/Off
2023-08-10T10:20:49.303 DEBUG:teuthology.orchestra.console.smithi074:pexpect command: ipmitool -H smithi074.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-10T10:20:49.322 INFO:teuthology.orchestra.console.smithi179:Power off
2023-08-10T10:20:49.323 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-10T10:20:49.333 DEBUG:teuthology.orchestra.console.smithi074:check power output: Chassis Power is off
2023-08-10T10:20:49.344 DEBUG:teuthology.orchestra.console.smithi179:power off output: Chassis Power Control: Down/Off
2023-08-10T10:20:49.344 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-10T10:20:49.364 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is off
2023-08-10T10:20:49.435 INFO:teuthology.orchestra.console.smithi074:Power off completed
2023-08-10T10:20:49.466 INFO:teuthology.orchestra.console.smithi179:Power off completed
2023-08-10T10:20:49.537 INFO:teuthology.orchestra.console.smithi074:Power on
2023-08-10T10:20:49.537 DEBUG:teuthology.orchestra.console.smithi074:pexpect command: ipmitool -H smithi074.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-08-10T10:20:49.559 DEBUG:teuthology.orchestra.console.smithi074:power on output: Chassis Power Control: Up/On
2023-08-10T10:20:49.560 DEBUG:teuthology.orchestra.console.smithi074:pexpect command: ipmitool -H smithi074.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-10T10:20:49.568 INFO:teuthology.orchestra.console.smithi179:Power on
2023-08-10T10:20:49.569 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-08-10T10:20:49.581 DEBUG:teuthology.orchestra.console.smithi074:check power output: Chassis Power is off
2023-08-10T10:20:49.588 DEBUG:teuthology.orchestra.console.smithi179:power on output: Chassis Power Control: Up/On
2023-08-10T10:20:49.589 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-10T10:20:49.609 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is off
2023-08-10T10:20:53.588 DEBUG:teuthology.orchestra.console.smithi074:pexpect command: ipmitool -H smithi074.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-10T10:20:53.611 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-10T10:20:53.697 DEBUG:teuthology.orchestra.console.smithi074:check power output: Chassis Power is off
2023-08-10T10:20:53.722 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on
2023-08-10T10:20:53.823 INFO:teuthology.orchestra.console.smithi179:Power on completed
2023-08-10T10:20:53.924 INFO:teuthology.provision.fog.smithi179:Waiting for deploy to finish
2023-08-10T10:20:53.924 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None
2023-08-10T10:20:57.700 DEBUG:teuthology.orchestra.console.smithi074:pexpect command: ipmitool -H smithi074.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-10T10:20:57.892 DEBUG:teuthology.orchestra.console.smithi074:check power output: Chassis Power is on
2023-08-10T10:20:57.993 INFO:teuthology.orchestra.console.smithi074:Power on completed
2023-08-10T10:20:58.094 INFO:teuthology.provision.fog.smithi074:Waiting for deploy to finish
2023-08-10T10:20:58.095 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:21:08.990 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None
2023-08-10T10:21:13.156 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:21:24.060 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None
2023-08-10T10:21:28.219 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:21:39.128 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None
2023-08-10T10:21:43.284 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:21:54.200 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None
2023-08-10T10:21:58.348 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:22:09.272 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None
2023-08-10T10:22:13.408 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:22:24.340 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None
2023-08-10T10:22:28.465 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:22:39.408 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None
2023-08-10T10:22:43.527 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:22:54.479 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None
2023-08-10T10:22:58.590 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:23:09.552 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None
2023-08-10T10:23:13.642 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:23:24.632 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None
2023-08-10T10:23:28.701 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:23:39.688 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None
2023-08-10T10:23:39.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-10T10:23:42.832 WARNING:teuthology.provision.fog.smithi179:[Errno None] Unable to connect to port 22 on 172.21.15.179
2023-08-10T10:23:43.756 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:23:48.836 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-10T10:23:51.888 WARNING:teuthology.provision.fog.smithi179:[Errno None] Unable to connect to port 22 on 172.21.15.179
2023-08-10T10:23:57.894 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-10T10:23:58.810 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:24:00.948 WARNING:teuthology.provision.fog.smithi179:[Errno None] Unable to connect to port 22 on 172.21.15.179
2023-08-10T10:24:06.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-10T10:24:10.036 WARNING:teuthology.provision.fog.smithi179:[Errno None] Unable to connect to port 22 on 172.21.15.179
2023-08-10T10:24:13.868 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:24:16.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-10T10:24:19.120 WARNING:teuthology.provision.fog.smithi179:[Errno None] Unable to connect to port 22 on 172.21.15.179
2023-08-10T10:24:25.124 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-10T10:24:28.180 WARNING:teuthology.provision.fog.smithi179:[Errno None] Unable to connect to port 22 on 172.21.15.179
2023-08-10T10:24:28.926 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:24:34.186 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-10T10:24:43.992 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:24:59.056 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:25:14.120 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:25:29.185 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:25:34.192 WARNING:teuthology.provision.fog.smithi179:timed out
2023-08-10T10:25:40.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-10T10:25:40.402 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-08-10T10:25:40.403 DEBUG:teuthology.orchestra.run.smithi179:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-08-10T10:25:40.668 INFO:teuthology.provision.fog.smithi179:Node is ready
2023-08-10T10:25:40.669 DEBUG:teuthology.orchestra.run.smithi179:> hostname
2023-08-10T10:25:40.689 INFO:teuthology.orchestra.run.smithi179.stdout:smithi179.front.sepia.ceph.com
2023-08-10T10:25:40.689 DEBUG:teuthology.orchestra.run.smithi179:> grep smithi179.front.sepia.ceph.com /etc/hosts
2023-08-10T10:25:40.748 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.179 smithi179.front.sepia.ceph.com smithi179
2023-08-10T10:25:40.749 DEBUG:teuthology.orchestra.run.smithi179:> sudo hostname smithi179
2023-08-10T10:25:40.856 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i -e 's/smithi179.front.sepia.ceph.com/smithi179/g' /etc/hosts
2023-08-10T10:25:40.885 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i -e 's/172.21.15.179/172.21.15.179/g' /etc/hosts
2023-08-10T10:25:40.952 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i -e 's/smithi179.front.sepia.ceph.com/smithi179/g' /etc/hostname
2023-08-10T10:25:41.020 DEBUG:teuthology.orchestra.run.smithi179:> sudo hostname smithi179
2023-08-10T10:25:41.088 DEBUG:teuthology.orchestra.run.smithi179:> cat /etc/os-release
2023-08-10T10:25:41.145 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="CentOS Stream"
2023-08-10T10:25:41.146 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="8"
2023-08-10T10:25:41.146 INFO:teuthology.orchestra.run.smithi179.stdout:ID="centos"
2023-08-10T10:25:41.146 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="rhel fedora"
2023-08-10T10:25:41.146 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="8"
2023-08-10T10:25:41.147 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el8"
2023-08-10T10:25:41.147 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="CentOS Stream 8"
2023-08-10T10:25:41.147 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31"
2023-08-10T10:25:41.147 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-08-10T10:25:41.147 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://centos.org/"
2023-08-10T10:25:41.147 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-08-10T10:25:41.147 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-08-10T10:25:41.148 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-08-10T10:25:41.148 INFO:teuthology.provision.fog.smithi179:Deploy complete!
2023-08-10T10:25:41.149 DEBUG:teuthology.parallel:result is None
2023-08-10T10:25:44.237 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:25:59.304 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:26:14.372 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:26:29.440 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:26:44.508 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:26:59.600 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:27:14.678 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:27:29.769 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:27:44.844 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:27:59.928 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:28:15.008 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:28:30.100 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:28:45.188 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:29:00.264 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:29:15.356 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:29:30.433 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:29:45.516 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:30:00.596 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:30:15.682 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:30:30.808 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:30:45.888 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:31:00.976 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:31:16.060 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:31:31.143 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:31:46.224 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:32:01.297 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:32:16.384 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:32:31.464 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:32:46.556 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:33:01.641 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:33:16.729 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:33:31.828 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:33:46.912 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:34:01.988 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:34:17.084 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:34:32.176 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:34:47.255 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:35:02.328 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:35:17.396 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:35:32.461 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:35:47.532 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:36:02.596 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:36:17.660 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:36:32.724 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:36:47.792 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:37:02.864 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:37:17.932 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:37:33.016 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:37:48.088 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:38:03.164 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:38:18.244 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:38:33.328 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:38:48.424 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:39:03.508 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:39:18.581 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:39:33.668 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:39:48.748 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:40:03.816 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:40:18.888 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:40:33.953 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:40:49.014 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:41:04.081 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:41:19.144 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:41:34.215 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:41:49.284 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:42:04.364 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:42:19.443 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:42:34.513 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:42:49.592 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:43:04.676 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:43:19.748 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:43:34.815 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:43:49.885 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:44:04.972 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:44:20.040 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:44:35.108 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:44:50.176 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:45:05.241 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:45:20.305 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:45:35.369 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:45:50.432 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:46:05.500 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:46:20.564 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:46:35.632 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:46:50.692 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:47:05.755 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:47:20.824 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:47:35.888 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:47:50.968 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:48:06.033 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:48:21.100 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:48:36.164 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:48:51.236 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:49:06.313 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:49:21.384 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:49:36.456 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:49:51.530 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:50:06.616 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:50:21.685 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:50:36.750 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:50:51.818 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:51:06.892 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:51:21.960 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:51:37.054 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:51:52.128 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:52:07.196 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:52:22.269 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:52:37.344 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:52:52.425 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:53:07.496 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:53:22.568 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:53:37.644 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:53:52.719 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:54:07.784 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:54:22.848 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:54:37.916 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:54:52.984 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:55:08.044 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:55:23.113 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:55:38.176 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:55:53.240 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:56:08.304 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:56:23.376 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:56:38.444 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:56:53.512 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:57:08.578 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:57:23.641 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:57:38.704 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:57:53.771 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:58:08.850 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:58:23.942 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:58:39.021 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:58:54.078 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:59:09.145 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:59:24.221 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:59:39.298 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T10:59:54.367 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:00:09.432 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:00:24.512 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:00:39.584 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:00:54.660 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:01:09.732 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:01:24.792 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:01:39.860 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:01:54.938 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:02:10.012 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:02:25.096 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:02:40.155 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:02:55.231 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:03:10.299 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:03:25.367 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:03:40.436 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:03:55.504 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:04:10.580 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:04:25.653 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:04:40.728 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:04:55.813 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:05:10.892 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:05:25.968 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:05:41.048 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:05:56.128 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:06:11.206 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:06:26.284 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:06:41.351 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:06:56.436 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:07:11.520 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:07:26.588 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:07:41.656 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:07:56.732 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:08:11.812 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:08:26.889 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:08:41.956 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:08:57.024 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:09:12.092 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:09:27.150 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:09:42.220 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:09:57.284 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:10:12.351 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:10:27.416 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:10:42.480 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:10:57.544 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:11:12.604 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:11:27.674 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:11:42.748 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:11:57.823 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:12:12.900 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:12:27.976 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:12:43.053 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:12:58.132 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:13:13.224 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:13:28.319 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:13:43.408 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:13:58.484 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:14:13.569 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:14:28.634 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:14:43.720 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:14:58.801 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:15:13.881 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:15:28.954 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:15:44.021 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:15:59.085 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:16:14.164 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:16:29.229 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:16:44.292 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:16:59.364 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:17:14.430 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:17:29.484 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:17:44.549 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:17:59.616 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:18:14.688 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:18:29.753 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:18:44.816 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:18:59.888 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:19:14.964 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:19:30.040 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:19:45.095 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:20:00.164 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:20:15.236 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:20:30.304 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:20:45.376 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:21:00.441 DEBUG:teuthology.provision.fog.smithi074:FOG request: GET /task/active None
2023-08-10T11:21:00.500 DEBUG:teuthology.provision.fog.smithi074:Canceling deploy task with ID 433413
2023-08-10T11:21:00.501 DEBUG:teuthology.provision.fog.smithi074:FOG request: DELETE /task/cancel {"id": 433413}
2023-08-10T11:21:00.551 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/provision/fog.py", line 86, in create
    self.wait_for_deploy_task(task_id)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/provision/fog.py", line 259, in wait_for_deploy_task
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (241) after waiting for 3600 seconds
2023-08-10T11:21:00.552 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-10T11:21:00.686 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=d604f5d5d891489fbc130f891567850c
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/provision/fog.py", line 86, in create
    self.wait_for_deploy_task(task_id)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/provision/fog.py", line 259, in wait_for_deploy_task
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (241) after waiting for 3600 seconds
2023-08-10T11:21:00.689 INFO:teuthology.nuke:Checking targets against current locks
2023-08-10T11:21:00.743 DEBUG:teuthology.nuke:shortname: smithi074
2023-08-10T11:21:00.744 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-08-10T11:21:00.766 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi074.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gsalomon-2023-08-10_10:17:39-rgw:verify-trino_tpcds_qe_fixes-distro-default-smithi/7364608', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-10 10:20:47.969551', 'locked_by': 'scheduled_gsalomon@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0='}
2023-08-10T11:21:00.769 DEBUG:teuthology.nuke:shortname: smithi179
2023-08-10T11:21:00.769 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-08-10T11:21:00.791 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gsalomon-2023-08-10_10:17:39-rgw:verify-trino_tpcds_qe_fixes-distro-default-smithi/7364608', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-10 10:20:47.970920', 'locked_by': 'scheduled_gsalomon@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}
2023-08-10T11:21:00.817 INFO:teuthology.orchestra.console.smithi074:Power off
2023-08-10T11:21:00.818 DEBUG:teuthology.orchestra.console.smithi074:pexpect command: ipmitool -H smithi074.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-10T11:21:00.839 DEBUG:teuthology.orchestra.console.smithi074:power off output: Chassis Power Control: Down/Off
2023-08-10T11:21:00.839 DEBUG:teuthology.orchestra.console.smithi074:pexpect command: ipmitool -H smithi074.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-10T11:21:00.853 INFO:teuthology.orchestra.console.smithi179:Power off
2023-08-10T11:21:00.854 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-10T11:21:00.862 DEBUG:teuthology.orchestra.console.smithi074:check power output: Chassis Power is on
2023-08-10T11:21:00.875 DEBUG:teuthology.orchestra.console.smithi179:power off output: Chassis Power Control: Down/Off
2023-08-10T11:21:00.876 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-10T11:21:00.896 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on
2023-08-10T11:21:04.868 DEBUG:teuthology.orchestra.console.smithi074:pexpect command: ipmitool -H smithi074.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-10T11:21:04.898 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-10T11:21:04.979 DEBUG:teuthology.orchestra.console.smithi074:check power output: Chassis Power is off
2023-08-10T11:21:05.008 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on
2023-08-10T11:21:05.081 INFO:teuthology.orchestra.console.smithi074:Power off completed
2023-08-10T11:21:05.232 INFO:teuthology.lock.ops:unlocked: smithi074.front.sepia.ceph.com
2023-08-10T11:21:09.012 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-10T11:21:09.121 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is off
2023-08-10T11:21:09.222 INFO:teuthology.orchestra.console.smithi179:Power off completed
2023-08-10T11:21:09.363 INFO:teuthology.lock.ops:unlocked: smithi179.front.sepia.ceph.com
2023-08-10T11:21:10.028 ERROR:teuthology.dispatcher.supervisor:Reimage failed 10 times ... marking machine down
2023-08-10T11:21:10.145 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/provision/fog.py", line 86, in create
    self.wait_for_deploy_task(task_id)
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/provision/fog.py", line 259, in wait_for_deploy_task
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (241) after waiting for 3600 seconds