2023-07-15T07:36:53.461 INFO:root:teuthology version: 0.0.1.dev147+gff9991b 2023-07-15T07:36:53.462 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T07:36:53.463 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T07:36:57.968 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f34ecdfb8b0>, signals=[15]) 2023-07-15T07:36:57.970 INFO:teuthology.lock.ops:Start node 'smithi031.front.sepia.ceph.com' reimaging 2023-07-15T07:36:57.970 INFO:teuthology.lock.ops:Updating [smithi031.front.sepia.ceph.com]: reset os type and version on server 2023-07-15T07:36:57.970 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2023-07-15T07:36:57.998 INFO:teuthology.lock.ops:Start node 'smithi122.front.sepia.ceph.com' reimaging 2023-07-15T07:36:57.999 INFO:teuthology.lock.ops:Updating [smithi122.front.sepia.ceph.com]: reset os type and version on server 2023-07-15T07:36:57.999 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2023-07-15T07:36:58.000 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /host {"name": "smithi031"} 2023-07-15T07:37:02.806 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /host {"name": "smithi122"} 2023-07-15T07:37:12.806 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /image {"name": "smithi_ubuntu_22.04"} 2023-07-15T07:37:22.846 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /image {"name": "smithi_ubuntu_22.04"} 2023-07-15T07:37:27.851 DEBUG:teuthology.provision.fog.smithi031:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-07-15T07:37:27.852 DEBUG:teuthology.provision.fog.smithi031:FOG request: PUT /host/136 {"imageID": 6} 2023-07-15T07:37:37.901 DEBUG:teuthology.provision.fog.smithi122:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-07-15T07:37:37.901 DEBUG:teuthology.provision.fog.smithi122:FOG request: PUT /host/227 {"imageID": 6} 2023-07-15T07:37:42.910 INFO:teuthology.provision.fog.smithi031:Scheduling deploy of ubuntu 22.04 2023-07-15T07:37:42.910 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-07-15T07:37:47.969 INFO:teuthology.provision.fog.smithi122:Scheduling deploy of ubuntu 22.04 2023-07-15T07:37:47.969 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None 2023-07-15T07:37:52.982 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /tasktype {"name": "deploy"} 2023-07-15T07:37:53.173 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /tasktype {"name": "deploy"} 2023-07-15T07:37:53.227 DEBUG:teuthology.provision.fog.smithi031:FOG request: POST /host/136/task {"taskTypeID": 1} 2023-07-15T07:37:58.234 DEBUG:teuthology.provision.fog.smithi122:FOG request: POST /host/227/task {"taskTypeID": 1} 2023-07-15T07:38:08.318 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None 2023-07-15T07:38:13.323 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-07-15T07:38:28.451 INFO:teuthology.orchestra.console:Power off smithi031 2023-07-15T07:38:28.453 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-15T07:38:28.475 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:38:28.494 INFO:teuthology.orchestra.console:Power off smithi122 2023-07-15T07:38:28.494 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-15T07:38:28.514 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:38:28.604 INFO:teuthology.orchestra.console:Power off for smithi031 completed 2023-07-15T07:38:28.635 INFO:teuthology.orchestra.console:Power off for smithi122 completed 2023-07-15T07:38:28.705 INFO:teuthology.orchestra.console:Power on smithi031 2023-07-15T07:38:28.706 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-15T07:38:28.729 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:38:28.738 INFO:teuthology.orchestra.console:Power on smithi122 2023-07-15T07:38:28.739 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-15T07:38:28.761 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:38:32.756 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:38:32.784 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:38:37.867 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:38:37.895 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:38:42.979 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:38:43.004 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:38:43.191 INFO:teuthology.orchestra.console:Power on for smithi031 completed 2023-07-15T07:38:43.292 INFO:teuthology.provision.fog.smithi031:Waiting for deploy to finish 2023-07-15T07:38:43.293 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-07-15T07:38:48.403 INFO:teuthology.orchestra.console:Power on for smithi122 completed 2023-07-15T07:38:48.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:38:48.504 INFO:teuthology.provision.fog.smithi122:Waiting for deploy to finish 2023-07-15T07:38:48.505 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None 2023-07-15T07:38:53.509 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-07-15T07:38:53.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:38:59.511 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:39:04.516 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-07-15T07:39:07.568 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-07-15T07:39:10.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:39:15.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:39:20.530 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-07-15T07:39:23.604 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-07-15T07:39:26.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:39:31.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:39:36.541 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-07-15T07:39:42.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:39:50.608 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-07-15T07:39:53.132 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-07-15T07:39:56.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:40:01.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:40:06.622 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-07-15T07:40:06.741 ERROR:teuthology.orchestra.connection:Error authenticating with smithi031.front.sepia.ceph.com: Authentication failed. 2023-07-15T07:40:06.744 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-15T07:40:06.746 DEBUG:teuthology.orchestra.run.smithi031:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-15T07:40:06.747 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:07.754 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:08.760 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:09.765 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:10.770 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:11.776 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:12.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:40:17.630 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:17.704 ERROR:teuthology.orchestra.connection:Error authenticating with smithi122.front.sepia.ceph.com: Authentication failed. 2023-07-15T07:40:17.704 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-15T07:40:17.705 DEBUG:teuthology.orchestra.run.smithi122:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-15T07:40:17.706 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:18.637 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:18.713 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:19.646 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:19.721 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:20.656 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:20.731 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:21.665 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:21.739 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:22.674 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:22.749 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:23.684 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:23.758 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:24.693 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:24.768 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:25.703 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:25.778 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:26.713 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:26.788 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:27.723 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:27.797 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:28.732 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:28.807 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:29.741 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:29.817 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:30.751 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:30.826 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:31.762 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:31.836 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:32.771 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:32.845 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:33.781 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:33.855 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:34.792 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:34.863 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:35.801 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:35.872 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:36.810 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:36.881 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:37.819 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:37.891 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:38.830 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:38.901 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:39.839 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:39.909 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:40.849 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:40.919 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:41.860 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:41.929 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:42.869 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:42.938 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:43.879 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:43.947 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:44.888 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:44.956 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:45.898 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:45.965 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:46.906 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:46.974 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:47.915 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:47.983 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:48.924 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:48.992 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:49.934 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:50.002 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:50.944 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:51.012 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:51.952 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:52.021 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:52.961 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:53.031 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:53.971 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:54.041 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:54.981 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:55.050 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:55.992 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:56.060 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:57.008 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:57.070 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:58.018 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:58.080 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:59.027 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:40:59.089 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:00.042 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:00.098 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:01.050 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:01.107 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:02.060 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:02.117 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:03.070 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:03.127 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:04.078 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:04.135 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:05.087 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:05.145 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:06.097 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:06.154 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:07.106 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:07.163 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:08.116 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:08.172 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:09.125 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:09.181 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:10.133 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:10.189 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:11.141 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:11.197 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:12.149 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:12.206 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:13.161 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:13.215 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:14.170 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:14.224 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:15.177 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:15.232 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:16.187 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:16.242 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:17.196 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:17.251 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:18.205 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:18.261 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:19.214 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:19.269 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:20.223 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:20.279 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:21.232 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:21.287 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:22.241 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:22.297 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:23.251 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:23.306 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:24.260 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:24.315 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:25.270 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:25.325 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:26.279 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:26.334 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:27.289 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:27.344 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:28.297 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:28.353 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:29.309 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:29.362 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:30.318 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:30.370 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:31.328 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:31.381 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:32.336 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:32.389 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:33.346 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:33.398 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:34.356 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:34.408 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:35.365 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:35.419 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:36.376 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:36.430 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:37.385 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:37.439 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:38.394 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:38.452 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:39.403 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:39.461 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:40.412 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:40.471 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:41.422 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:41.481 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:42.431 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:42.489 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:43.440 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:43.499 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:44.449 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:44.508 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:45.459 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:45.517 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:46.468 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:46.526 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:47.476 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:47.534 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:48.484 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:48.543 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:49.494 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:49.553 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:50.503 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:50.562 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:51.512 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:51.571 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:52.522 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:52.580 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:53.531 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:53.588 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:54.539 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:54.597 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:55.548 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:55.605 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:56.556 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:56.614 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:57.565 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:57.623 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:58.575 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:58.632 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:59.584 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:41:59.640 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:00.592 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:00.650 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:01.601 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:01.658 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:02.609 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:02.666 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:03.617 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:03.675 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:04.626 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:04.684 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:05.635 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:05.693 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:06.627 ERROR:teuthology.provision.fog: on smithi031 2023-07-15T07:42:06.700 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:07.705 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:08.711 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:09.716 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:10.722 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:11.728 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:12.735 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:13.740 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:14.745 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:15.629 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-07-15T07:42:15.629 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:42:20.635 ERROR:teuthology.provision.fog: on smithi122 2023-07-15T07:42:20.751 ERROR:teuthology.orchestra.connection:Error authenticating with smithi031.front.sepia.ceph.com: Authentication failed. 2023-07-15T07:42:20.752 DEBUG:teuthology.orchestra.run.smithi031:> true 2023-07-15T07:42:20.752 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:42:29.636 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2023-07-15T07:42:29.637 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:42:34.719 ERROR:teuthology.orchestra.connection:Error authenticating with smithi122.front.sepia.ceph.com: Authentication failed. 2023-07-15T07:42:34.720 DEBUG:teuthology.orchestra.run.smithi122:> true 2023-07-15T07:42:34.720 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:44:29.641 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-07-15T07:44:29.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:44:34.760 ERROR:teuthology.orchestra.connection:Error authenticating with smithi031.front.sepia.ceph.com: Authentication failed. 2023-07-15T07:44:34.761 DEBUG:teuthology.orchestra.run.smithi031:> true 2023-07-15T07:44:34.762 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:44:43.648 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2023-07-15T07:44:43.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:44:43.766 ERROR:teuthology.orchestra.connection:Error authenticating with smithi122.front.sepia.ceph.com: Authentication failed. 2023-07-15T07:44:43.766 DEBUG:teuthology.orchestra.run.smithi122:> true 2023-07-15T07:44:43.767 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:46:46.653 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-07-15T07:46:46.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:46:51.776 ERROR:teuthology.orchestra.connection:Error authenticating with smithi031.front.sepia.ceph.com: Authentication failed. 2023-07-15T07:46:51.776 DEBUG:teuthology.orchestra.run.smithi031:> true 2023-07-15T07:46:51.777 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:46:55.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2023-07-15T07:46:55.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:46:55.773 ERROR:teuthology.orchestra.connection:Error authenticating with smithi122.front.sepia.ceph.com: Authentication failed. 2023-07-15T07:46:55.774 DEBUG:teuthology.orchestra.run.smithi122:> true 2023-07-15T07:46:55.775 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:48:54.666 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-07-15T07:48:54.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:48:54.787 ERROR:teuthology.orchestra.connection:Error authenticating with smithi031.front.sepia.ceph.com: Authentication failed. 2023-07-15T07:48:54.788 DEBUG:teuthology.orchestra.run.smithi031:> true 2023-07-15T07:48:54.789 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:48:58.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2023-07-15T07:48:58.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T07:49:03.784 ERROR:teuthology.orchestra.connection:Error authenticating with smithi122.front.sepia.ceph.com: Authentication failed. 2023-07-15T07:49:03.786 DEBUG:teuthology.orchestra.run.smithi122:> true 2023-07-15T07:49:03.787 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-15T07:50:54.673 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-15T07:50:54.676 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/provision/fog.py", line 299, in _wait_for_ready self.remote.run(args=cmd, timeout=600) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi031 2023-07-15T07:50:54.676 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T07:50:54.677 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T07:50:59.771 INFO:teuthology.nuke:Checking targets against current locks 2023-07-15T07:51:02.142 DEBUG:teuthology.nuke:shortname: smithi031 2023-07-15T07:51:02.142 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-15T07:51:02.177 DEBUG:teuthology.nuke:shortname: smithi122 2023-07-15T07:51:02.177 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-15T07:51:02.185 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-14_23:39:06-rados-wip-yuri5-testing-2023-07-14-0757-distro-default-smithi/7338772', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-15 07:36:52.667970', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0='} 2023-07-15T07:51:02.211 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-14_23:39:06-rados-wip-yuri5-testing-2023-07-14-0757-distro-default-smithi/7338772', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-15 07:36:52.669742', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-15T07:51:02.249 INFO:teuthology.orchestra.console:Power off smithi031 2023-07-15T07:51:02.250 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-15T07:51:02.279 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:51:02.300 INFO:teuthology.orchestra.console:Power off smithi122 2023-07-15T07:51:02.301 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-15T07:51:02.333 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:51:03.673 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-15T07:51:06.315 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:51:06.363 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:51:11.428 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:51:11.481 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:51:11.693 INFO:teuthology.orchestra.console:Power off for smithi122 completed 2023-07-15T07:51:21.834 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:51:21.877 INFO:teuthology.lock.ops:unlocked: smithi122.front.sepia.ceph.com 2023-07-15T07:51:26.945 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:51:32.060 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:51:37.172 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:51:42.285 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:51:47.397 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:51:52.511 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T07:51:52.723 INFO:teuthology.orchestra.console:Power off for smithi031 completed 2023-07-15T07:51:52.887 INFO:teuthology.lock.ops:unlocked: smithi031.front.sepia.ceph.com 2023-07-15T07:51:53.153 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/dispatcher/supervisor.py", line 57, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/provision/fog.py", line 299, in _wait_for_ready self.remote.run(args=cmd, timeout=600) File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi031