2023-09-28T20:43:43.864 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-09-28T20:43:43.864 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-28T20:43:44.000 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f23a48ecaf0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f23a53d7670>, signals=[15])
2023-09-28T20:43:44.000 INFO:teuthology.lock.ops:Start node 'smithi114.front.sepia.ceph.com' reimaging
2023-09-28T20:43:44.001 INFO:teuthology.lock.ops:Updating [smithi114.front.sepia.ceph.com]: reset os type and version on server
2023-09-28T20:43:44.001 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server
2023-09-28T20:43:44.022 INFO:teuthology.lock.ops:Start node 'smithi158.front.sepia.ceph.com' reimaging
2023-09-28T20:43:44.023 INFO:teuthology.lock.ops:Updating [smithi158.front.sepia.ceph.com]: reset os type and version on server
2023-09-28T20:43:44.023 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server
2023-09-28T20:43:44.024 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /host {"name": "smithi114"}
2023-09-28T20:43:44.047 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /host {"name": "smithi158"}
2023-09-28T20:43:44.107 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /image {"name": "smithi_rhel_8.4"}
2023-09-28T20:43:44.112 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /image {"name": "smithi_rhel_8.4"}
2023-09-28T20:43:44.152 DEBUG:teuthology.provision.fog.smithi158:Requesting image smithi_rhel_8.4 (ID 7)
2023-09-28T20:43:44.152 DEBUG:teuthology.provision.fog.smithi158:FOG request: PUT /host/263 {"imageID": 7}
2023-09-28T20:43:44.157 DEBUG:teuthology.provision.fog.smithi114:Requesting image smithi_rhel_8.4 (ID 7)
2023-09-28T20:43:44.157 DEBUG:teuthology.provision.fog.smithi114:FOG request: PUT /host/219 {"imageID": 7}
2023-09-28T20:43:44.199 INFO:teuthology.provision.fog.smithi114:Scheduling deploy of rhel 8.4
2023-09-28T20:43:44.199 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:43:44.210 INFO:teuthology.provision.fog.smithi158:Scheduling deploy of rhel 8.4
2023-09-28T20:43:44.211 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-09-28T20:43:44.262 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /tasktype {"name": "deploy"}
2023-09-28T20:43:44.271 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /tasktype {"name": "deploy"}
2023-09-28T20:43:44.297 DEBUG:teuthology.provision.fog.smithi114:FOG request: POST /host/219/task {"taskTypeID": 1}
2023-09-28T20:43:44.309 DEBUG:teuthology.provision.fog.smithi158:FOG request: POST /host/263/task {"taskTypeID": 1}
2023-09-28T20:43:44.449 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:43:44.472 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-09-28T20:43:44.540 INFO:teuthology.orchestra.console.smithi114:Power off
2023-09-28T20:43:44.540 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-28T20:43:44.558 INFO:teuthology.orchestra.console.smithi158:Power off
2023-09-28T20:43:44.558 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-28T20:43:44.568 DEBUG:teuthology.orchestra.console.smithi114:power off output: Chassis Power Control: Down/Off
2023-09-28T20:43:44.569 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-28T20:43:44.583 DEBUG:teuthology.orchestra.console.smithi158:power off output: Chassis Power Control: Down/Off
2023-09-28T20:43:44.583 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-28T20:43:44.596 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off
2023-09-28T20:43:44.609 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off
2023-09-28T20:43:44.697 INFO:teuthology.orchestra.console.smithi114:Power off completed
2023-09-28T20:43:44.709 INFO:teuthology.orchestra.console.smithi158:Power off completed
2023-09-28T20:43:44.798 INFO:teuthology.orchestra.console.smithi114:Power on
2023-09-28T20:43:44.798 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-28T20:43:44.811 INFO:teuthology.orchestra.console.smithi158:Power on
2023-09-28T20:43:44.811 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-28T20:43:44.823 DEBUG:teuthology.orchestra.console.smithi114:power on output: Chassis Power Control: Up/On
2023-09-28T20:43:44.824 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-28T20:43:44.831 DEBUG:teuthology.orchestra.console.smithi158:power on output: Chassis Power Control: Up/On
2023-09-28T20:43:44.832 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-28T20:43:44.844 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off
2023-09-28T20:43:44.852 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off
2023-09-28T20:43:48.846 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-28T20:43:48.853 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-28T20:43:48.954 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on
2023-09-28T20:43:48.960 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on
2023-09-28T20:43:49.056 INFO:teuthology.orchestra.console.smithi114:Power on completed
2023-09-28T20:43:49.061 INFO:teuthology.orchestra.console.smithi158:Power on completed
2023-09-28T20:43:49.157 INFO:teuthology.provision.fog.smithi114:Waiting for deploy to finish
2023-09-28T20:43:49.157 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:43:49.162 INFO:teuthology.provision.fog.smithi158:Waiting for deploy to finish
2023-09-28T20:43:49.162 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-09-28T20:44:04.246 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-09-28T20:44:04.247 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:44:19.325 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-09-28T20:44:19.327 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:44:34.400 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-09-28T20:44:34.401 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:44:49.458 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:44:49.459 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-09-28T20:45:04.534 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:45:04.535 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-09-28T20:45:19.626 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:45:19.627 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-09-28T20:45:34.710 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:45:34.711 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-09-28T20:45:49.794 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-09-28T20:45:49.795 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:46:04.882 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-09-28T20:46:04.883 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:46:19.966 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-09-28T20:46:19.967 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:46:35.046 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-09-28T20:46:35.048 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:46:50.134 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:46:50.135 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-09-28T20:47:05.226 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None
2023-09-28T20:47:05.227 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:47:05.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-28T20:47:20.314 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:47:23.770 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-09-28T20:47:29.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-28T20:47:32.826 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-09-28T20:47:35.402 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:47:38.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-28T20:47:41.886 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-09-28T20:47:47.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-28T20:47:50.509 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:47:50.970 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-09-28T20:47:56.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-28T20:48:00.026 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-09-28T20:48:05.595 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:48:06.027 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-28T20:48:09.083 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-09-28T20:48:15.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-28T20:48:18.138 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-09-28T20:48:20.685 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:48:24.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-28T20:48:35.774 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:48:42.622 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-09-28T20:48:48.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-28T20:48:50.845 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:48:51.710 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-09-28T20:48:57.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-28T20:49:00.798 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2023-09-28T20:49:05.930 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:49:06.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-28T20:49:07.041 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-09-28T20:49:07.042 DEBUG:teuthology.orchestra.run.smithi158:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-09-28T20:49:21.002 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:49:27.358 INFO:teuthology.provision.fog.smithi158:Node is ready
2023-09-28T20:49:27.358 DEBUG:teuthology.orchestra.run.smithi158:> hostname
2023-09-28T20:49:27.413 INFO:teuthology.orchestra.run.smithi158.stdout:smithi158.front.sepia.ceph.com
2023-09-28T20:49:27.414 DEBUG:teuthology.orchestra.run.smithi158:> grep smithi158.front.sepia.ceph.com /etc/hosts
2023-09-28T20:49:27.467 INFO:teuthology.orchestra.run.smithi158.stdout:172.21.15.158 smithi158.front.sepia.ceph.com smithi158
2023-09-28T20:49:27.468 DEBUG:teuthology.orchestra.run.smithi158:> sudo hostname smithi158
2023-09-28T20:49:27.600 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/smithi158.front.sepia.ceph.com/smithi158/g' /etc/hosts
2023-09-28T20:49:27.624 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/172.21.15.158/172.21.15.158/g' /etc/hosts
2023-09-28T20:49:27.689 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/smithi158.front.sepia.ceph.com/smithi158/g' /etc/hostname
2023-09-28T20:49:27.752 DEBUG:teuthology.orchestra.run.smithi158:> sudo hostname smithi158
2023-09-28T20:49:27.814 DEBUG:teuthology.orchestra.run.smithi158:> cat /etc/os-release
2023-09-28T20:49:27.868 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="Red Hat Enterprise Linux"
2023-09-28T20:49:27.869 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="8.4 (Ootpa)"
2023-09-28T20:49:27.869 INFO:teuthology.orchestra.run.smithi158.stdout:ID="rhel"
2023-09-28T20:49:27.869 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="fedora"
2023-09-28T20:49:27.869 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="8.4"
2023-09-28T20:49:27.869 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el8"
2023-09-28T20:49:27.869 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-09-28T20:49:27.869 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31"
2023-09-28T20:49:27.869 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-09-28T20:49:27.870 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://www.redhat.com/"
2023-09-28T20:49:27.870 INFO:teuthology.orchestra.run.smithi158.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-09-28T20:49:27.870 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-09-28T20:49:27.870 INFO:teuthology.orchestra.run.smithi158.stdout:
2023-09-28T20:49:27.870 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-09-28T20:49:27.870 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-09-28T20:49:27.870 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-09-28T20:49:27.870 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-09-28T20:49:27.871 INFO:teuthology.provision.fog.smithi158:Deploy complete!
2023-09-28T20:49:27.872 DEBUG:teuthology.parallel:result is None
2023-09-28T20:49:36.086 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:49:51.178 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:50:06.270 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:50:21.350 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:50:36.434 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:50:51.513 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:51:06.598 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:51:21.683 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:51:36.758 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:51:51.830 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:52:06.902 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:52:21.960 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:52:37.038 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:52:52.110 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:53:07.178 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:53:22.247 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:53:37.330 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:53:52.410 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:54:07.497 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:54:22.586 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:54:37.662 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:54:52.742 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:55:07.826 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:55:22.906 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:55:37.979 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:55:53.050 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:56:08.134 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:56:23.208 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:56:38.286 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:56:53.354 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:57:08.418 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:57:23.490 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:57:38.563 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:57:53.644 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:58:09.377 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:58:24.694 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:58:39.774 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:58:54.856 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:59:09.941 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:59:25.023 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:59:40.102 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T20:59:55.173 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:00:10.250 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:00:25.335 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:00:40.418 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:00:55.506 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:01:10.574 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:01:25.642 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:01:40.714 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:01:55.778 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:02:10.846 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:02:25.911 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:02:40.983 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:02:56.052 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:03:11.122 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:03:26.196 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:03:41.270 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:03:56.350 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:04:11.426 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:04:26.510 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:04:41.594 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:04:56.674 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:05:11.756 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:05:26.836 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:05:41.915 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:05:57.203 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:06:12.302 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:06:27.382 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:06:42.470 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:06:57.610 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:07:12.694 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:07:27.762 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:07:42.830 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:07:57.898 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:08:12.966 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:08:28.074 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:08:43.142 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:08:58.211 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:09:13.278 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:09:28.350 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:09:43.407 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:09:58.470 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:10:13.629 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:10:28.699 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:10:43.757 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:10:58.830 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:11:14.030 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:11:29.243 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:11:44.314 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:11:59.388 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:12:14.462 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:12:29.538 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:12:44.615 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:12:59.702 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:13:14.774 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:13:29.846 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:13:44.914 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:13:59.982 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:14:15.058 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:14:30.138 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:14:45.210 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:15:00.282 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:15:15.354 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:15:30.426 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:15:45.498 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:16:00.582 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:16:15.654 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:16:30.722 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:16:45.794 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:17:00.866 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:17:15.930 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:17:30.998 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:17:46.066 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:18:01.136 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:18:16.206 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:18:31.278 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:18:46.342 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:19:01.402 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:19:16.477 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:19:31.547 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:19:46.934 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:20:02.257 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:20:17.330 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:20:32.418 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:20:47.498 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:21:02.594 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:21:17.694 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:21:32.786 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:21:47.866 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:22:02.983 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:22:18.262 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:22:33.342 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:22:48.422 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:23:03.510 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:23:18.597 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:23:33.742 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:23:48.932 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:24:04.018 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:24:19.094 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:24:34.390 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:24:49.490 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:25:04.582 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:25:19.662 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:25:34.745 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:25:49.856 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:26:04.953 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:26:20.042 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:26:35.142 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:26:50.211 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:27:05.290 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:27:20.363 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:27:35.438 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:27:50.500 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:28:05.566 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:28:20.650 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:28:35.750 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:28:50.834 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:29:05.915 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:29:21.058 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:29:36.126 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:29:51.258 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:30:06.346 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:30:21.438 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:30:36.547 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:30:51.614 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:31:06.702 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:31:21.878 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:31:36.970 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:31:52.034 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:32:07.138 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:32:22.218 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:32:39.590 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:32:54.660 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:33:09.729 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:33:24.794 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:33:39.858 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:33:54.922 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:34:09.989 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:34:25.062 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:34:40.126 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:34:55.198 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:35:10.540 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:35:25.618 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:35:40.690 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:35:55.754 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:36:10.814 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:36:25.914 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:36:40.978 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:36:56.038 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:37:11.098 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:37:26.182 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:37:41.246 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:37:56.382 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:38:11.454 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:38:26.526 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:38:41.610 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:38:56.690 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:39:11.774 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:39:26.882 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:39:41.984 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:39:57.098 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:40:12.182 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:40:27.256 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:40:42.326 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:40:57.391 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:41:12.454 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:41:27.518 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:41:42.578 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:41:57.645 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:42:12.706 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:42:27.768 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:42:42.826 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:42:57.888 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:43:13.014 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:43:28.086 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:43:43.154 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:43:58.230 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-09-28T21:43:58.282 DEBUG:teuthology.provision.fog.smithi114:Canceling deploy task with ID 507119
2023-09-28T21:43:58.282 DEBUG:teuthology.provision.fog.smithi114:FOG request: DELETE /task/cancel {"id": 507119}
2023-09-28T21:43:58.587 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 86, in create
    self.wait_for_deploy_task(task_id)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 259, in wait_for_deploy_task
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (241) after waiting for 3600 seconds
2023-09-28T21:43:58.599 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-28T21:43:58.743 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=1f4cac0c68a049c0bdc10ef1ef29e625
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 86, in create
    self.wait_for_deploy_task(task_id)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 259, in wait_for_deploy_task
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (241) after waiting for 3600 seconds
2023-09-28T21:43:58.745 INFO:teuthology.nuke:Checking targets against current locks
2023-09-28T21:43:58.816 DEBUG:teuthology.nuke:shortname: smithi114
2023-09-28T21:43:58.816 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-28T21:43:58.839 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-27_19:29:17-rados-wip-yuri6-testing-2023-09-27-0938-quincy-distro-default-smithi/7404084', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-28 20:43:43.073168', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}
2023-09-28T21:43:58.843 DEBUG:teuthology.nuke:shortname: smithi158
2023-09-28T21:43:58.843 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-28T21:43:58.864 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi158.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-27_19:29:17-rados-wip-yuri6-testing-2023-09-27-0938-quincy-distro-default-smithi/7404084', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-28 20:43:43.074566', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}
2023-09-28T21:43:58.888 INFO:teuthology.orchestra.console.smithi114:Power off
2023-09-28T21:43:58.889 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-28T21:43:58.913 DEBUG:teuthology.orchestra.console.smithi114:power off output: Chassis Power Control: Down/Off
2023-09-28T21:43:58.921 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-28T21:43:58.936 INFO:teuthology.orchestra.console.smithi158:Power off
2023-09-28T21:43:58.942 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-28T21:43:58.952 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off
2023-09-28T21:43:58.981 DEBUG:teuthology.orchestra.console.smithi158:power off output: Chassis Power Control: Down/Off
2023-09-28T21:43:58.984 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-28T21:43:59.012 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on
2023-09-28T21:43:59.079 INFO:teuthology.orchestra.console.smithi114:Power off completed
2023-09-28T21:43:59.284 INFO:teuthology.lock.ops:unlocked: smithi114.front.sepia.ceph.com
2023-09-28T21:44:03.046 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-28T21:44:03.187 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on
2023-09-28T21:44:07.228 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-28T21:44:07.369 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off
2023-09-28T21:44:07.486 INFO:teuthology.orchestra.console.smithi158:Power off completed
2023-09-28T21:44:07.641 INFO:teuthology.lock.ops:unlocked: smithi158.front.sepia.ceph.com
2023-09-28T21:44:07.979 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 86, in create
    self.wait_for_deploy_task(task_id)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 259, in wait_for_deploy_task
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (241) after waiting for 3600 seconds