2024-03-22T02:11:06.370 INFO:root:teuthology version: 0.0.1.dev262+ge691533
2024-03-22T02:11:06.370 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-22T02:11:06.503 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f2848c7af40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f2849cdbc10>, signals=[15])
2024-03-22T02:11:06.504 INFO:teuthology.lock.ops:Start node 'smithi072.front.sepia.ceph.com' reimaging
2024-03-22T02:11:06.504 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: reset os type and version on server
2024-03-22T02:11:06.504 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server
2024-03-22T02:11:06.523 INFO:teuthology.lock.ops:Start node 'smithi089.front.sepia.ceph.com' reimaging
2024-03-22T02:11:06.523 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: reset os type and version on server
2024-03-22T02:11:06.523 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server
2024-03-22T02:11:06.639 DEBUG:teuthology.provision.fog.smithi072:Requesting image smithi_centos_9.stream (ID 10)
2024-03-22T02:11:06.680 DEBUG:teuthology.provision.fog.smithi089:Requesting image smithi_centos_9.stream (ID 10)
2024-03-22T02:11:06.690 INFO:teuthology.provision.fog.smithi072:Scheduling deploy of centos 9.stream
2024-03-22T02:11:06.727 INFO:teuthology.provision.fog.smithi089:Scheduling deploy of centos 9.stream
2024-03-22T02:11:06.934 INFO:teuthology.orchestra.console.smithi072:Power off
2024-03-22T02:11:06.934 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-22T02:11:06.957 DEBUG:teuthology.orchestra.console.smithi072:power off output: Chassis Power Control: Down/Off
2024-03-22T02:11:06.957 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-22T02:11:06.976 INFO:teuthology.orchestra.console.smithi089:Power off
2024-03-22T02:11:06.976 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-22T02:11:06.985 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off
2024-03-22T02:11:06.998 DEBUG:teuthology.orchestra.console.smithi089:power off output: Chassis Power Control: Down/Off
2024-03-22T02:11:06.998 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-22T02:11:07.023 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off
2024-03-22T02:11:07.086 INFO:teuthology.orchestra.console.smithi072:Power off completed
2024-03-22T02:11:07.124 INFO:teuthology.orchestra.console.smithi089:Power off completed
2024-03-22T02:11:07.188 INFO:teuthology.orchestra.console.smithi072:Power on
2024-03-22T02:11:07.188 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-22T02:11:07.211 DEBUG:teuthology.orchestra.console.smithi072:power on output: Chassis Power Control: Up/On
2024-03-22T02:11:07.211 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-22T02:11:07.225 INFO:teuthology.orchestra.console.smithi089:Power on
2024-03-22T02:11:07.225 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-22T02:11:07.234 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off
2024-03-22T02:11:07.247 DEBUG:teuthology.orchestra.console.smithi089:power on output: Chassis Power Control: Up/On
2024-03-22T02:11:07.248 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-22T02:11:07.270 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off
2024-03-22T02:11:11.238 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-22T02:11:11.271 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-22T02:11:11.348 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off
2024-03-22T02:11:11.381 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on
2024-03-22T02:11:11.481 INFO:teuthology.orchestra.console.smithi089:Power on completed
2024-03-22T02:11:11.582 INFO:teuthology.provision.fog.smithi089:Waiting for deploy to finish
2024-03-22T02:11:15.352 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-22T02:11:15.461 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on
2024-03-22T02:11:15.562 INFO:teuthology.orchestra.console.smithi072:Power on completed
2024-03-22T02:11:15.663 INFO:teuthology.provision.fog.smithi072:Waiting for deploy to finish
2024-03-22T02:13:57.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:14:00.546 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89
2024-03-22T02:14:06.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:14:09.634 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89
2024-03-22T02:14:15.639 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:14:16.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:14:18.718 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89
2024-03-22T02:14:19.614 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:14:24.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:14:25.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:14:27.806 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89
2024-03-22T02:14:28.670 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:14:33.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:14:34.672 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:14:36.898 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89
2024-03-22T02:14:37.726 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:14:42.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:14:43.729 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:14:45.986 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89
2024-03-22T02:14:46.782 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:14:51.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:14:52.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:14:55.838 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:15:01.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:15:04.926 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:15:10.366 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89
2024-03-22T02:15:10.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:15:13.982 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:15:16.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:15:19.422 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89
2024-03-22T02:15:19.984 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:15:25.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:15:28.482 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89
2024-03-22T02:15:34.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:15:34.754 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-22T02:15:34.755 DEBUG:teuthology.orchestra.run.smithi089:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-22T02:15:50.258 INFO:teuthology.provision.fog.smithi089:Node is ready
2024-03-22T02:15:50.258 DEBUG:teuthology.orchestra.run.smithi089:> hostname
2024-03-22T02:15:50.276 INFO:teuthology.orchestra.run.smithi089.stdout:smithi046
2024-03-22T02:15:50.276 DEBUG:teuthology.orchestra.run.smithi089:> grep smithi046 /etc/hosts
2024-03-22T02:15:50.334 DEBUG:teuthology.orchestra.run:got remote process result: 1
2024-03-22T02:15:50.335 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i -e 's/smithi046/smithi089/g' /etc/hostname
2024-03-22T02:15:50.459 DEBUG:teuthology.orchestra.run.smithi089:> sudo hostname smithi089
2024-03-22T02:15:50.487 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release
2024-03-22T02:15:50.542 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream"
2024-03-22T02:15:50.542 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="9"
2024-03-22T02:15:50.542 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos"
2024-03-22T02:15:50.542 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora"
2024-03-22T02:15:50.542 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="9"
2024-03-22T02:15:50.542 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el9"
2024-03-22T02:15:50.542 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-22T02:15:50.542 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31"
2024-03-22T02:15:50.542 INFO:teuthology.orchestra.run.smithi089.stdout:LOGO="fedora-logo-icon"
2024-03-22T02:15:50.542 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-22T02:15:50.542 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/"
2024-03-22T02:15:50.543 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-22T02:15:50.543 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-22T02:15:50.543 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-22T02:15:50.543 INFO:teuthology.provision.fog.smithi089:Deploy complete!
2024-03-22T02:15:50.545 DEBUG:teuthology.parallel:result is None
2024-03-22T02:16:20.015 WARNING:teuthology.provision.fog.smithi072:timed out
2024-03-22T02:16:26.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:16:44.578 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:16:50.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:16:53.662 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:16:59.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:17:02.750 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:17:08.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:17:11.806 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:17:17.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:17:20.866 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:17:26.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:17:29.950 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:17:35.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:17:39.010 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:17:45.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:17:48.094 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:17:54.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:17:57.150 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:18:03.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:18:06.206 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:18:12.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:18:15.294 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:18:21.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:18:24.350 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:18:30.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:18:33.406 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:18:39.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:18:42.462 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:18:48.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:18:51.554 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:18:57.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:19:00.638 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:19:06.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:19:09.698 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:19:15.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:19:18.786 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:19:24.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:19:27.874 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:19:33.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:19:36.962 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:19:42.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:19:46.046 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:19:52.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:19:55.102 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:20:01.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:20:04.162 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:20:10.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:20:13.246 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:20:19.253 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:20:22.338 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:20:28.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:20:31.422 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:20:37.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:20:40.478 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:20:46.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:20:49.534 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:20:55.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:20:58.626 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:21:04.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:21:07.710 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:21:13.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:21:16.770 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:21:22.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:21:25.858 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:21:31.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:21:34.946 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:21:40.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:21:44.030 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:21:50.037 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:21:53.122 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:21:59.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:22:02.206 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:22:08.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:22:11.262 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:22:17.266 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:22:20.318 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:22:26.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:22:29.406 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:22:35.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:22:38.462 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:22:44.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:22:47.522 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:22:53.527 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:22:56.610 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:23:02.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:23:05.698 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:23:11.704 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:23:14.786 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:23:20.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:23:23.870 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:23:29.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:23:32.926 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:23:38.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:23:41.986 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:23:47.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:23:51.074 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:23:57.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:24:00.126 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:24:06.132 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:24:09.182 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:24:15.189 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:24:18.270 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:24:24.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:24:27.326 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:24:33.333 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:24:36.414 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:24:42.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:24:45.470 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:24:51.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:24:54.558 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:25:00.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:25:03.614 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:25:09.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:25:12.706 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:25:18.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:25:21.794 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:25:27.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:25:30.878 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:25:36.883 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:25:39.938 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:25:45.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:25:49.022 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:25:55.029 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:25:58.110 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:26:04.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:26:07.166 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:26:13.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:26:16.226 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:26:22.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:26:25.310 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:26:31.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:26:34.370 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:26:40.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:26:43.454 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:26:49.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:26:52.510 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:26:58.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:27:01.570 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:27:07.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:27:10.654 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:27:16.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:27:19.711 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:27:25.717 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:27:28.798 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:27:34.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:27:37.854 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:27:43.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:27:46.910 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:27:52.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:27:55.973 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:28:01.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:28:05.054 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:28:11.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:28:14.113 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:28:20.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:28:23.202 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:28:29.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:28:32.290 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:28:38.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:28:41.374 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:28:47.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:28:50.462 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:28:56.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:28:59.550 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:29:05.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:29:08.642 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:29:14.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:29:17.726 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:29:23.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:29:26.782 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:29:32.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:29:35.838 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:29:41.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:29:44.926 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:29:50.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:29:53.982 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:29:59.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:30:03.042 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:30:09.047 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:30:12.126 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:30:18.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:30:21.182 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:30:27.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-22T02:30:30.238 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72
2024-03-22T02:30:30.240 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/fog.py", line 275, in _wait_for_ready
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds
2024-03-22T02:30:30.242 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-22T02:30:30.544 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=988d07ac58804cbf85c15c76ac0e961f
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/fog.py", line 275, in _wait_for_ready
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds
2024-03-22T02:30:30.546 INFO:teuthology.nuke:Checking targets against current locks
2024-03-22T02:30:30.604 DEBUG:teuthology.nuke:shortname: smithi072
2024-03-22T02:30:30.606 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-03-22T02:30:30.625 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615811', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 02:11:05.237305', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKo+S2CXMsjpVcPBFgBBxourTVomk7ZnFioj/wZwp2FQnk+p+7YpjlIcrH0cdg8qzENQcY7AY/FnSyIKX1KDZzE='}
2024-03-22T02:30:30.630 DEBUG:teuthology.nuke:shortname: smithi089
2024-03-22T02:30:30.630 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-03-22T02:30:30.653 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-22_01:06:30-fs-wip-batrick-testing-20240321.172515-distro-default-smithi/7615811', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 02:11:05.238765', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLWzifK/OZv43YSON/Qai1svwB4TjTJhvPiWpjHja5YIoIpzEAE5MhByrLzxAxpJXLlwFKb3r5VESzqMTnqdIrI='}
2024-03-22T02:30:30.677 INFO:teuthology.orchestra.console.smithi072:Power off
2024-03-22T02:30:30.677 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-22T02:30:30.701 DEBUG:teuthology.orchestra.console.smithi072:power off output: Chassis Power Control: Down/Off
2024-03-22T02:30:30.701 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-22T02:30:30.716 INFO:teuthology.orchestra.console.smithi089:Power off
2024-03-22T02:30:30.717 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-22T02:30:30.725 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on
2024-03-22T02:30:30.738 DEBUG:teuthology.orchestra.console.smithi089:power off output: Chassis Power Control: Down/Off
2024-03-22T02:30:30.738 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-22T02:30:30.759 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on
2024-03-22T02:30:34.730 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-22T02:30:34.760 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-22T02:30:34.841 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on
2024-03-22T02:30:34.870 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on
2024-03-22T02:30:38.845 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-22T02:30:38.871 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-22T02:30:38.958 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off
2024-03-22T02:30:38.980 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off
2024-03-22T02:30:39.059 INFO:teuthology.orchestra.console.smithi072:Power off completed
2024-03-22T02:30:39.081 INFO:teuthology.orchestra.console.smithi089:Power off completed
2024-03-22T02:30:39.206 INFO:teuthology.lock.ops:unlocked: smithi072.front.sepia.ceph.com
2024-03-22T02:30:39.231 INFO:teuthology.lock.ops:unlocked: smithi089.front.sepia.ceph.com
2024-03-22T02:30:39.523 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/fog.py", line 275, in _wait_for_ready
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds