2024-01-31T20:38:04.270 INFO:root:teuthology version: 0.0.1.dev253+g6128cc3
2024-01-31T20:38:04.271 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-31T20:38:04.421 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fc085726f40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fc08678cc10>, signals=[15])
2024-01-31T20:38:04.421 INFO:teuthology.lock.ops:Start node 'smithi140.front.sepia.ceph.com' reimaging
2024-01-31T20:38:04.422 INFO:teuthology.lock.ops:Updating [smithi140.front.sepia.ceph.com]: reset os type and version on server
2024-01-31T20:38:04.422 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server
2024-01-31T20:38:04.441 INFO:teuthology.lock.ops:Start node 'smithi164.front.sepia.ceph.com' reimaging
2024-01-31T20:38:04.441 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: reset os type and version on server
2024-01-31T20:38:04.441 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server
2024-01-31T20:38:04.571 DEBUG:teuthology.provision.fog.smithi140:Requesting image smithi_centos_9.stream (ID 10)
2024-01-31T20:38:04.576 DEBUG:teuthology.provision.fog.smithi164:Requesting image smithi_centos_9.stream (ID 10)
2024-01-31T20:38:04.611 INFO:teuthology.provision.fog.smithi140:Scheduling deploy of centos 9.stream
2024-01-31T20:38:04.633 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of centos 9.stream
2024-01-31T20:38:04.911 INFO:teuthology.orchestra.console.smithi140:Power off
2024-01-31T20:38:04.911 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-31T20:38:04.932 DEBUG:teuthology.orchestra.console.smithi140:power off output: Chassis Power Control: Down/Off
2024-01-31T20:38:04.932 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-31T20:38:04.953 INFO:teuthology.orchestra.console.smithi164:Power off
2024-01-31T20:38:04.953 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-31T20:38:04.960 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off
2024-01-31T20:38:04.972 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off
2024-01-31T20:38:04.972 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-31T20:38:04.992 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2024-01-31T20:38:05.062 INFO:teuthology.orchestra.console.smithi140:Power off completed
2024-01-31T20:38:05.093 INFO:teuthology.orchestra.console.smithi164:Power off completed
2024-01-31T20:38:05.163 INFO:teuthology.orchestra.console.smithi140:Power on
2024-01-31T20:38:05.163 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-31T20:38:05.184 DEBUG:teuthology.orchestra.console.smithi140:power on output: Chassis Power Control: Up/On
2024-01-31T20:38:05.184 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-31T20:38:05.194 INFO:teuthology.orchestra.console.smithi164:Power on
2024-01-31T20:38:05.194 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-31T20:38:05.207 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off
2024-01-31T20:38:05.213 DEBUG:teuthology.orchestra.console.smithi164:power on output: Chassis Power Control: Up/On
2024-01-31T20:38:05.213 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-31T20:38:05.235 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2024-01-31T20:38:09.208 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-31T20:38:09.236 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-31T20:38:09.317 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off
2024-01-31T20:38:09.344 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2024-01-31T20:38:13.323 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-31T20:38:13.345 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-31T20:38:13.433 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on
2024-01-31T20:38:13.453 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2024-01-31T20:38:13.534 INFO:teuthology.orchestra.console.smithi140:Power on completed
2024-01-31T20:38:13.554 INFO:teuthology.orchestra.console.smithi164:Power on completed
2024-01-31T20:38:13.635 INFO:teuthology.provision.fog.smithi140:Waiting for deploy to finish
2024-01-31T20:38:13.656 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish
2024-01-31T20:40:44.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:40:44.865 ERROR:teuthology.orchestra.connection:Error authenticating with smithi164.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-01-31T20:40:59.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:41:02.720 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:41:08.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:41:11.804 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:41:17.805 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:41:20.864 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:41:26.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:41:29.948 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:41:35.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:41:39.004 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:41:45.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:41:48.060 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:41:48.867 WARNING:teuthology.provision.fog.smithi164:timed out
2024-01-31T20:41:54.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:41:54.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:42:54.064 WARNING:teuthology.provision.fog.smithi140:timed out
2024-01-31T20:42:54.871 WARNING:teuthology.provision.fog.smithi164:timed out
2024-01-31T20:43:00.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:43:00.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:43:01.139 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-31T20:43:01.140 DEBUG:teuthology.orchestra.run.smithi164:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-31T20:43:01.488 INFO:teuthology.provision.fog.smithi164:Node is ready
2024-01-31T20:43:01.489 DEBUG:teuthology.orchestra.run.smithi164:> hostname
2024-01-31T20:43:01.504 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164.front.sepia.ceph.com
2024-01-31T20:43:01.505 DEBUG:teuthology.orchestra.run.smithi164:> grep smithi164.front.sepia.ceph.com /etc/hosts
2024-01-31T20:43:01.560 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164 smithi164.front.sepia.ceph.com smithi164
2024-01-31T20:43:01.560 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164
2024-01-31T20:43:01.666 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164.front.sepia.ceph.com/smithi164/g' /etc/hosts
2024-01-31T20:43:01.694 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/172.21.15.164/172.21.15.164/g' /etc/hosts
2024-01-31T20:43:01.762 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164.front.sepia.ceph.com/smithi164/g' /etc/hostname
2024-01-31T20:43:01.829 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164
2024-01-31T20:43:01.896 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release
2024-01-31T20:43:01.950 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream"
2024-01-31T20:43:01.950 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9"
2024-01-31T20:43:01.950 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos"
2024-01-31T20:43:01.950 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora"
2024-01-31T20:43:01.951 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9"
2024-01-31T20:43:01.951 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9"
2024-01-31T20:43:01.951 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-31T20:43:01.951 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31"
2024-01-31T20:43:01.951 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon"
2024-01-31T20:43:01.951 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-31T20:43:01.951 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/"
2024-01-31T20:43:01.951 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-31T20:43:01.951 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-31T20:43:01.951 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-31T20:43:01.952 INFO:teuthology.provision.fog.smithi164:Deploy complete!
2024-01-31T20:43:01.953 DEBUG:teuthology.parallel:result is None
2024-01-31T20:43:18.464 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:43:24.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:43:27.516 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:43:33.517 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:43:36.576 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:43:42.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:43:45.628 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:43:51.632 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:43:54.688 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:44:00.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:44:03.776 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:44:09.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:44:12.828 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:44:18.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:44:21.888 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:44:27.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:44:30.940 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:44:36.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:44:40.000 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:44:46.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:44:49.052 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:44:55.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:44:58.108 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:45:04.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:45:07.164 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:45:13.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:45:16.224 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:45:22.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:45:25.276 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:45:31.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:45:34.332 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:45:40.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:45:43.392 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:45:49.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:45:52.448 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:45:58.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:46:01.500 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:46:07.502 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:46:10.556 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:46:16.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:46:19.612 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:46:25.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:46:28.668 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:46:34.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:46:37.724 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:46:43.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:46:46.780 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:46:52.781 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:46:55.840 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:47:01.841 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:47:04.892 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:47:10.894 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:47:13.948 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:47:19.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:47:23.004 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:47:29.005 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:47:32.064 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:47:38.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:47:41.116 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:47:47.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:47:50.172 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:47:56.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:47:59.232 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:48:05.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:48:08.284 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:48:14.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:48:17.344 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:48:23.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:48:26.400 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:48:32.402 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:48:35.452 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:48:41.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:48:44.508 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:48:50.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:48:53.568 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:48:59.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:49:02.652 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:49:08.653 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:49:11.708 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:49:17.710 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:49:20.764 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:49:26.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:49:29.820 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:49:35.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:49:38.876 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:49:44.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:49:47.932 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:49:53.934 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:49:56.992 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:50:02.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:50:06.044 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:50:12.045 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:50:15.100 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:50:21.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:50:24.156 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:50:30.158 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:50:33.212 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:50:39.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:50:42.268 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:50:48.270 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:50:51.324 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:50:57.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:51:00.380 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:51:06.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:51:09.436 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:51:15.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:51:18.492 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:51:24.494 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:51:27.552 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:51:33.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:51:36.607 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:51:42.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:51:45.660 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:51:51.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:51:54.716 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:52:00.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:52:03.772 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:52:09.773 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:52:12.828 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:52:18.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:52:21.884 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:52:27.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:52:30.940 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:52:36.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:52:40.000 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:52:46.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:52:49.052 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:52:55.053 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:52:58.112 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:53:04.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:53:07.196 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:53:13.197 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:53:16.252 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:53:22.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:53:25.308 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:53:31.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:53:34.368 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:53:40.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:53:43.420 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:53:49.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:53:52.480 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:53:58.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:54:01.564 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:54:07.566 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:54:10.620 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:54:16.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:54:19.676 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:54:25.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:54:28.736 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:54:34.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:54:37.788 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:54:43.789 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:54:46.844 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:54:52.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:54:55.904 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:55:01.905 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:55:04.956 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:55:10.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:55:14.012 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:55:20.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:55:23.068 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:55:29.069 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:55:32.128 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:55:38.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:55:41.180 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:55:47.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:55:50.236 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:55:56.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:55:59.292 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:56:05.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:56:08.348 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:56:14.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:56:17.436 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:56:23.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:56:26.492 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:56:32.493 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:56:35.548 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:56:41.549 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:56:44.604 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:56:50.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:56:53.664 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:56:59.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:57:02.716 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:57:08.717 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-31T20:57:11.776 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140
2024-01-31T20:57:11.778 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/provision/fog.py", line 275, in _wait_for_ready
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds
2024-01-31T20:57:11.780 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-31T20:57:12.074 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=d787d028db4d4048801dd7a156c0a543
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/provision/fog.py", line 275, in _wait_for_ready
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds
2024-01-31T20:57:12.077 INFO:teuthology.nuke:Checking targets against current locks
2024-01-31T20:57:12.148 DEBUG:teuthology.nuke:shortname: smithi140
2024-01-31T20:57:12.148 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-31T20:57:12.172 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi140.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-30_16:08:52-rados-wip-yuri3-testing-2024-01-29-1434-distro-default-smithi/7539101', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-31 20:38:03.169720', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBPdY5RIb9OSOZMnS/DlxgLG6ZuRFXF47dhl+Rj9ZqsEbWUOxO8InLE5bAmz7emNLm2W/lbidnuyqvdcfL7aEb8='}
2024-01-31T20:57:12.175 DEBUG:teuthology.nuke:shortname: smithi164
2024-01-31T20:57:12.175 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-31T20:57:12.201 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-30_16:08:52-rados-wip-yuri3-testing-2024-01-29-1434-distro-default-smithi/7539101', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-31 20:38:03.167646', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBExbTLLMTTWNOHwhJjOTS34M5nRcjIHzdZQa5T9xJNnzIYpf6zDC8yipwf/4dgRU9P7iwBzz0hCmmQguwssoDqw='}
2024-01-31T20:57:12.228 INFO:teuthology.orchestra.console.smithi140:Power off
2024-01-31T20:57:12.228 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-31T20:57:12.250 DEBUG:teuthology.orchestra.console.smithi140:power off output: Chassis Power Control: Down/Off
2024-01-31T20:57:12.251 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-31T20:57:12.268 INFO:teuthology.orchestra.console.smithi164:Power off
2024-01-31T20:57:12.269 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-31T20:57:12.279 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on
2024-01-31T20:57:12.289 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off
2024-01-31T20:57:12.289 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-31T20:57:12.311 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2024-01-31T20:57:16.280 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-31T20:57:16.313 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-31T20:57:16.390 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on
2024-01-31T20:57:16.424 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2024-01-31T20:57:20.392 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-31T20:57:20.424 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-31T20:57:20.501 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off
2024-01-31T20:57:20.534 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2024-01-31T20:57:20.602 INFO:teuthology.orchestra.console.smithi140:Power off completed
2024-01-31T20:57:20.753 INFO:teuthology.lock.ops:unlocked: smithi140.front.sepia.ceph.com
2024-01-31T20:57:24.535 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-31T20:57:24.645 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2024-01-31T20:57:24.746 INFO:teuthology.orchestra.console.smithi164:Power off completed
2024-01-31T20:57:24.896 INFO:teuthology.lock.ops:unlocked: smithi164.front.sepia.ceph.com
2024-01-31T20:57:25.182 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/provision/fog.py", line 275, in _wait_for_ready
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_6128cc3ecb49b7d62475e3595041c19b5326ca7c/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds