2023-07-06T04:00:26.203 INFO:root:teuthology version: 0.0.1.dev140+gb53f236
2023-07-06T04:00:26.204 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-06T04:00:26.205 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-06T04:00:40.314 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f19246fc640>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f1924079670>, signals=[15])
2023-07-06T04:00:40.315 INFO:teuthology.lock.ops:Start node 'smithi046.front.sepia.ceph.com' reimaging
2023-07-06T04:00:40.316 INFO:teuthology.lock.ops:Updating [smithi046.front.sepia.ceph.com]: reset os type and version on server
2023-07-06T04:00:40.316 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server
2023-07-06T04:00:40.337 INFO:teuthology.lock.ops:Start node 'smithi073.front.sepia.ceph.com' reimaging
2023-07-06T04:00:40.338 INFO:teuthology.lock.ops:Updating [smithi073.front.sepia.ceph.com]: reset os type and version on server
2023-07-06T04:00:40.338 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server
2023-07-06T04:00:40.339 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /host {"name": "smithi046"}
2023-07-06T04:00:44.753 INFO:teuthology.lock.ops:Start node 'smithi194.front.sepia.ceph.com' reimaging
2023-07-06T04:00:44.754 INFO:teuthology.lock.ops:Updating [smithi194.front.sepia.ceph.com]: reset os type and version on server
2023-07-06T04:00:44.754 INFO:teuthology.lock.ops:Updating smithi194.front.sepia.ceph.com on lock server
2023-07-06T04:00:44.755 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /host {"name": "smithi073"}
2023-07-06T04:01:04.762 DEBUG:teuthology.provision.fog.smithi194:FOG request: GET /host {"name": "smithi194"}
2023-07-06T04:01:19.783 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-07-06T04:01:29.797 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-07-06T04:01:29.826 DEBUG:teuthology.provision.fog.smithi194:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-07-06T04:01:29.841 DEBUG:teuthology.provision.fog.smithi073:Requesting image smithi_centos_8.stream (ID 3)
2023-07-06T04:01:29.841 DEBUG:teuthology.provision.fog.smithi073:FOG request: PUT /host/178 {"imageID": 3}
2023-07-06T04:01:29.854 DEBUG:teuthology.provision.fog.smithi046:Requesting image smithi_centos_8.stream (ID 3)
2023-07-06T04:01:29.855 DEBUG:teuthology.provision.fog.smithi046:FOG request: PUT /host/151 {"imageID": 3}
2023-07-06T04:01:29.876 DEBUG:teuthology.provision.fog.smithi194:Requesting image smithi_centos_8.stream (ID 3)
2023-07-06T04:01:29.877 DEBUG:teuthology.provision.fog.smithi194:FOG request: PUT /host/299 {"imageID": 3}
2023-07-06T04:01:29.880 INFO:teuthology.provision.fog.smithi073:Scheduling deploy of centos 8.stream
2023-07-06T04:01:29.880 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-07-06T04:01:29.901 INFO:teuthology.provision.fog.smithi046:Scheduling deploy of centos 8.stream
2023-07-06T04:01:29.901 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None
2023-07-06T04:01:29.918 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /tasktype {"name": "deploy"}
2023-07-06T04:01:29.923 INFO:teuthology.provision.fog.smithi194:Scheduling deploy of centos 8.stream
2023-07-06T04:01:29.923 DEBUG:teuthology.provision.fog.smithi194:FOG request: GET /task/active None
2023-07-06T04:01:29.945 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /tasktype {"name": "deploy"}
2023-07-06T04:01:29.952 DEBUG:teuthology.provision.fog.smithi073:FOG request: POST /host/178/task {"taskTypeID": 1}
2023-07-06T04:01:29.961 DEBUG:teuthology.provision.fog.smithi194:FOG request: GET /tasktype {"name": "deploy"}
2023-07-06T04:01:29.981 DEBUG:teuthology.provision.fog.smithi046:FOG request: POST /host/151/task {"taskTypeID": 1}
2023-07-06T04:01:30.001 DEBUG:teuthology.provision.fog.smithi194:FOG request: POST /host/299/task {"taskTypeID": 1}
2023-07-06T04:01:30.004 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-07-06T04:01:30.027 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None
2023-07-06T04:01:35.063 DEBUG:teuthology.provision.fog.smithi194:FOG request: GET /task/active None
2023-07-06T04:01:45.107 INFO:teuthology.orchestra.console:Power off smithi046
2023-07-06T04:01:45.108 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-06T04:01:45.124 INFO:teuthology.orchestra.console:Power off smithi073
2023-07-06T04:01:45.125 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-06T04:01:45.139 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:01:45.150 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:01:45.179 INFO:teuthology.orchestra.console:Power off smithi194
2023-07-06T04:01:45.179 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-06T04:01:45.199 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:01:49.168 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:01:49.176 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:01:49.221 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:01:53.283 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:01:53.291 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:01:53.331 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:01:53.494 INFO:teuthology.orchestra.console:Power off for smithi046 completed
2023-07-06T04:01:53.501 INFO:teuthology.orchestra.console:Power off for smithi073 completed
2023-07-06T04:01:53.596 INFO:teuthology.orchestra.console:Power on smithi046
2023-07-06T04:01:53.596 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-07-06T04:01:53.604 INFO:teuthology.orchestra.console:Power on smithi073
2023-07-06T04:01:53.605 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-07-06T04:01:53.617 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:01:53.628 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:01:57.446 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:01:57.640 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:01:57.650 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:02:02.557 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:02:02.751 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:02:02.763 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:02:07.668 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:02:07.863 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:02:07.874 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:02:12.782 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:02:12.974 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:02:12.986 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:02:13.197 INFO:teuthology.orchestra.console:Power on for smithi073 completed
2023-07-06T04:02:13.298 INFO:teuthology.provision.fog.smithi073:Waiting for deploy to finish
2023-07-06T04:02:13.298 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-07-06T04:02:17.897 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:02:18.086 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:02:18.297 INFO:teuthology.orchestra.console:Power on for smithi046 completed
2023-07-06T04:02:18.398 INFO:teuthology.provision.fog.smithi046:Waiting for deploy to finish
2023-07-06T04:02:18.398 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None
2023-07-06T04:02:18.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-06T04:02:21.520 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46
2023-07-06T04:02:23.008 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:02:27.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-06T04:02:32.529 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:02:32.537 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-07-06T04:02:37.545 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46
2023-07-06T04:02:37.656 INFO:teuthology.orchestra.console:Power off for smithi194 completed
2023-07-06T04:02:37.757 INFO:teuthology.orchestra.console:Power on smithi194
2023-07-06T04:02:37.757 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-07-06T04:02:43.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-06T04:02:51.636 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46
2023-07-06T04:02:52.604 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-07-06T04:02:57.614 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:02:57.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-06T04:03:00.720 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46
2023-07-06T04:03:02.627 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:03:06.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-06T04:03:07.739 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:03:12.673 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-07-06T04:03:17.681 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:03:22.791 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:03:25.168 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46
2023-07-06T04:03:27.904 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:03:31.172 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-06T04:03:36.177 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-07-06T04:03:36.179 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:03:37.851 ERROR:teuthology.orchestra.connection:Error authenticating with smithi046.front.sepia.ceph.com: Authentication failed.
2023-07-06T04:03:37.853 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-06T04:03:37.854 DEBUG:teuthology.orchestra.run.smithi046:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-06T04:03:37.855 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:03:38.861 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:03:39.867 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:03:40.874 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:03:41.880 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:03:42.776 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:03:42.887 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:03:43.893 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:03:44.900 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:03:45.905 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:03:46.911 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:03:47.888 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:03:47.917 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:03:48.922 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:03:49.928 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:03:50.934 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:03:51.939 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:03:52.844 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-07-06T04:03:57.849 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:03:57.859 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:03:58.866 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:03:59.872 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:00.883 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:01.890 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:02.897 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:02.961 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:04:03.904 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:04.912 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:05.918 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:06.928 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:07.935 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:08.074 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:04:08.942 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:09.950 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:10.956 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:11.961 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:12.913 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-07-06T04:04:17.921 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:04:17.932 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:18.939 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:19.945 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:20.951 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:21.958 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:22.964 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:23.033 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:04:23.972 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:24.978 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:25.983 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:26.990 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:27.995 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:28.146 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:04:29.002 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-06T04:04:29.256 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-07-06T04:04:29.257 DEBUG:teuthology.provision.fog.smithi194:Canceling deploy task with ID None
2023-07-06T04:04:29.257 DEBUG:teuthology.provision.fog.smithi194:FOG request: DELETE /task/cancel {"id": None}
2023-07-06T04:04:34.260 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-07-06T04:04:38.433 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-06T04:04:43.439 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/dispatcher/supervisor.py", line 227, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/console.py", line 243, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi194
2023-07-06T04:04:43.440 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-06T04:04:43.440 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-06T04:04:48.449 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-07-06T04:04:48.533 INFO:teuthology.nuke:Checking targets against current locks
2023-07-06T04:04:52.969 DEBUG:teuthology.nuke:shortname: smithi046
2023-07-06T04:04:52.970 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-07-06T04:04:52.993 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327936', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 04:00:25.350646', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoWW8w+N5xlB83QDeOhs5QHBR+3FhSbJdjQzLEoDF5yPq50lNykycRTRpkrfuxFrP45myRKA1gpe+Y2dXBLYUQ4RgijdM0mscTxynvygTSEDUKVjXAbB8PNrHxervT063+Xe324RytJNfWz9A3tDhf2DpUcil0vliMFBSlc9rkfbt/q2npszEzUeQobAhPa5YJ8Wm0hU2IbK1TIclVz2C8eZhQI0wrrB28/jV/D/D95kgCHE44dNnU0/cz63RooS7lVcThLyuynLPt8xY8eajveuhqW4Nx6Rc6ghbxCdY2ZeTIHzxSeBnEEysGOj9AyMc1AdaLkYWopTdOpmV9hlRxpCes4TtWgPOprsC/JLiuquYe0sZSS3/9IMUpaJhHtOGWuy0vv+/ULH8oEgpbCNEKIv03Wt3QL5OzZUiSyd9RTfGc8ly0QgiVCrZt9zr0AS65M1blFuDZtoKrb183cKpv9cyqP5TAqd2DuAHpxmB85popZpw0dQTgbuBGYkhaPcM='}
2023-07-06T04:04:52.995 DEBUG:teuthology.nuke:shortname: smithi073
2023-07-06T04:04:52.995 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-07-06T04:04:53.016 DEBUG:teuthology.nuke:shortname: smithi194
2023-07-06T04:04:53.017 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-07-06T04:04:53.022 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327936', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 04:00:25.351899', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4ykq8zmwhHmcBijIuWIFAAZdWp7uHCc9r4lgVC/1nu50gcDEy6uJ31Mto1c0yUoefuoA9sXWhGg0+itMM+1R6WZ/d91LbqVOTvGy2yS98Dop3Yw3T91xoaCDSWXMG4GarKiCOdThIb8HAayHb2TiZW7Pk9hqynOpk9VJpetno1ksGpUItFx5R/QR4DXWb+G5vF01Yr2Vyjpq0iA4S2GXfgUnROPYmIyhEnW2U6Iw6AOa5xIj/6nSuQLjcZII8JZx4wHrMo+K7T9cjRV4yyUuPpHwnha3LLNr/rrWF1XOC7GwCHzFqHZskRKnjbS2iKKgYnBD2k5MzUFH+I/DidqggqTFrhqk9bsSHMBHjJcB4nRA3z6LdC6DQFphsBoCvA0Jl5Ij40HjKWFNtvI00wiexomWXoXm5bBVJTfRd/xzZcXol9sXnmPfaYzbvZWG6znzfxbcAoJZ4KSDwGkdTexm7rb22TKU+bQh+4G2VrouVCyIhhN1r+sMcBtJ8k4NON8M='}
2023-07-06T04:04:53.045 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi194.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/7327936', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 04:00:25.349144', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMbKtJUxhtPlwHho5AoCriwlb++7AzaBjLO0Q4A1RWyHClEkdGkxgO3ipHaSKUKOSuV/hMx1m/5XMfd2OcHy+MQ='}
2023-07-06T04:04:53.056 INFO:teuthology.orchestra.console:Power off smithi046
2023-07-06T04:04:53.057 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-06T04:04:53.079 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:04:53.095 INFO:teuthology.orchestra.console:Power off smithi073
2023-07-06T04:04:53.096 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-06T04:04:53.118 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:04:53.139 INFO:teuthology.orchestra.console:Power off smithi194
2023-07-06T04:04:53.139 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-06T04:04:53.163 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:04:54.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-06T04:04:59.457 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:04:59.466 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:04:59.476 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:05:02.512 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-07-06T04:05:04.569 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:05:04.579 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:05:04.590 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:05:08.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-06T04:05:13.521 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:05:13.530 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:05:13.541 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-06T04:05:13.744 INFO:teuthology.orchestra.console:Power off for smithi073 completed
2023-07-06T04:05:18.853 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-07-06T04:05:23.748 INFO:teuthology.orchestra.console:Power off for smithi046 completed
2023-07-06T04:05:23.748 INFO:teuthology.orchestra.console:Power off for smithi194 completed
2023-07-06T04:05:23.776 INFO:teuthology.lock.ops:unlocked: smithi073.front.sepia.ceph.com
2023-07-06T04:05:23.898 INFO:teuthology.lock.ops:unlocked: smithi194.front.sepia.ceph.com
2023-07-06T04:05:23.907 INFO:teuthology.lock.ops:unlocked: smithi046.front.sepia.ceph.com
2023-07-06T04:05:24.310 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/dispatcher/supervisor.py", line 57, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/dispatcher/supervisor.py", line 227, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/console.py", line 243, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi194