2023-07-15T07:29:34.052 INFO:root:teuthology version: 0.0.1.dev147+gff9991b
2023-07-15T07:29:34.053 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-15T07:29:34.053 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-15T07:29:53.655 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f0c0307a610>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f0c042968b0>, signals=[15])
2023-07-15T07:29:53.657 INFO:teuthology.lock.ops:Start node 'smithi016.front.sepia.ceph.com' reimaging
2023-07-15T07:29:53.657 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: reset os type and version on server
2023-07-15T07:29:53.657 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server
2023-07-15T07:30:08.784 INFO:teuthology.lock.ops:Start node 'smithi081.front.sepia.ceph.com' reimaging
2023-07-15T07:30:08.784 INFO:teuthology.lock.ops:Updating [smithi081.front.sepia.ceph.com]: reset os type and version on server
2023-07-15T07:30:08.785 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server
2023-07-15T07:30:08.786 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /host {"name": "smithi016"}
2023-07-15T07:30:13.741 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /host {"name": "smithi081"}
2023-07-15T07:30:13.789 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /image {"name": "smithi_ubuntu_20.04"}
2023-07-15T07:30:13.807 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /image {"name": "smithi_ubuntu_20.04"}
2023-07-15T07:30:13.846 DEBUG:teuthology.provision.fog.smithi016:Requesting image smithi_ubuntu_20.04 (ID 2)
2023-07-15T07:30:13.847 DEBUG:teuthology.provision.fog.smithi016:FOG request: PUT /host/122 {"imageID": 2}
2023-07-15T07:30:13.853 DEBUG:teuthology.provision.fog.smithi081:Requesting image smithi_ubuntu_20.04 (ID 2)
2023-07-15T07:30:13.853 DEBUG:teuthology.provision.fog.smithi081:FOG request: PUT /host/186 {"imageID": 2}
2023-07-15T07:30:13.893 INFO:teuthology.provision.fog.smithi016:Scheduling deploy of ubuntu 20.04
2023-07-15T07:30:13.893 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-07-15T07:30:13.897 INFO:teuthology.provision.fog.smithi081:Scheduling deploy of ubuntu 20.04
2023-07-15T07:30:13.897 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-07-15T07:30:13.947 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /tasktype {"name": "deploy"}
2023-07-15T07:30:13.953 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /tasktype {"name": "deploy"}
2023-07-15T07:30:13.976 DEBUG:teuthology.provision.fog.smithi016:FOG request: POST /host/122/task {"taskTypeID": 1}
2023-07-15T07:30:13.992 DEBUG:teuthology.provision.fog.smithi081:FOG request: POST /host/186/task {"taskTypeID": 1}
2023-07-15T07:30:14.026 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-07-15T07:30:14.046 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-07-15T07:30:14.116 INFO:teuthology.orchestra.console:Power off smithi016
2023-07-15T07:30:14.118 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-15T07:30:14.137 INFO:teuthology.orchestra.console:Power off smithi081
2023-07-15T07:30:14.137 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-15T07:30:14.147 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:30:14.158 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:30:14.269 INFO:teuthology.orchestra.console:Power off for smithi016 completed
2023-07-15T07:30:14.280 INFO:teuthology.orchestra.console:Power off for smithi081 completed
2023-07-15T07:30:14.372 INFO:teuthology.orchestra.console:Power on smithi016
2023-07-15T07:30:14.372 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-07-15T07:30:14.382 INFO:teuthology.orchestra.console:Power on smithi081
2023-07-15T07:30:14.383 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-07-15T07:30:14.396 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:30:14.407 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:30:18.424 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:30:18.432 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:30:23.535 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:30:23.545 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:30:28.648 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:30:28.657 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:30:28.868 INFO:teuthology.orchestra.console:Power on for smithi081 completed
2023-07-15T07:30:28.969 INFO:teuthology.provision.fog.smithi081:Waiting for deploy to finish
2023-07-15T07:30:28.969 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-07-15T07:30:33.760 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:30:38.872 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:30:43.984 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:30:44.023 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-07-15T07:30:49.097 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:30:49.308 INFO:teuthology.orchestra.console:Power on for smithi016 completed
2023-07-15T07:30:49.409 INFO:teuthology.provision.fog.smithi016:Waiting for deploy to finish
2023-07-15T07:30:49.409 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-07-15T07:30:59.088 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-07-15T07:31:04.467 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-07-15T07:31:19.162 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-07-15T07:31:24.522 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-07-15T07:31:39.232 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-07-15T07:31:44.368 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-07-15T07:31:54.296 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-07-15T07:32:04.444 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-07-15T07:32:09.500 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-07-15T07:32:23.177 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-07-15T07:32:29.576 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-07-15T07:32:43.273 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-07-15T07:32:49.673 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-07-15T07:32:58.392 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-07-15T07:33:03.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-15T07:33:03.546 ERROR:teuthology.orchestra.connection:Error authenticating with smithi016.front.sepia.ceph.com: Authentication failed.
2023-07-15T07:33:03.549 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-15T07:33:03.550 DEBUG:teuthology.orchestra.run.smithi016:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-15T07:33:03.550 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:04.556 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:05.561 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:06.567 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:07.573 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:08.578 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:09.584 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:09.756 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-07-15T07:33:14.761 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:15.767 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:16.772 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:17.777 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:18.783 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:19.788 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:20.796 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:21.802 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:22.809 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:23.814 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:24.824 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:25.829 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:26.835 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:27.841 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:28.846 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:29.839 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-07-15T07:33:33.498 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:34.506 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:35.511 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:36.516 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:37.521 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:38.526 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:39.531 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:40.536 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:41.541 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:33:48.592 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-07-15T07:34:03.685 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-07-15T07:34:05.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-15T07:34:13.492 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-07-15T07:34:19.494 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-15T07:34:27.568 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-07-15T07:34:33.569 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-15T07:34:41.652 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-07-15T07:34:47.653 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-15T07:34:55.728 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-07-15T07:35:01.730 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-15T07:35:20.176 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-07-15T07:35:26.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-15T07:35:33.555 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-15T07:35:33.556 DEBUG:teuthology.orchestra.run.smithi081:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-15T07:35:55.421 INFO:teuthology.provision.fog.smithi081:Node is ready
2023-07-15T07:35:55.422 DEBUG:teuthology.orchestra.run.smithi081:> hostname
2023-07-15T07:35:55.429 INFO:teuthology.orchestra.run.smithi081.stdout:smithi081.front.sepia.ceph.com
2023-07-15T07:35:55.429 DEBUG:teuthology.orchestra.run.smithi081:> grep smithi081.front.sepia.ceph.com /etc/hosts
2023-07-15T07:35:55.436 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81 smithi081.front.sepia.ceph.com smithi081
2023-07-15T07:35:55.437 DEBUG:teuthology.orchestra.run.smithi081:> sudo hostname smithi081
2023-07-15T07:35:55.517 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i -e 's/smithi081.front.sepia.ceph.com/smithi081/g' /etc/hosts
2023-07-15T07:35:55.535 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i -e 's/172.21.15.81/172.21.15.81/g' /etc/hosts
2023-07-15T07:35:55.545 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i -e 's/smithi081.front.sepia.ceph.com/smithi081/g' /etc/hostname
2023-07-15T07:35:55.558 DEBUG:teuthology.orchestra.run.smithi081:> sudo hostname smithi081
2023-07-15T07:35:55.567 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release
2023-07-15T07:35:55.573 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="Ubuntu"
2023-07-15T07:35:55.574 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="20.04.4 LTS (Focal Fossa)"
2023-07-15T07:35:55.574 INFO:teuthology.orchestra.run.smithi081.stdout:ID=ubuntu
2023-07-15T07:35:55.574 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE=debian
2023-07-15T07:35:55.575 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS"
2023-07-15T07:35:55.575 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="20.04"
2023-07-15T07:35:55.575 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://www.ubuntu.com/"
2023-07-15T07:35:55.576 INFO:teuthology.orchestra.run.smithi081.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-07-15T07:35:55.576 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-07-15T07:35:55.576 INFO:teuthology.orchestra.run.smithi081.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-07-15T07:35:55.577 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_CODENAME=focal
2023-07-15T07:35:55.577 INFO:teuthology.orchestra.run.smithi081.stdout:UBUNTU_CODENAME=focal
2023-07-15T07:35:55.578 INFO:teuthology.provision.fog.smithi081:Deploy complete!
2023-07-15T07:35:55.579 DEBUG:teuthology.parallel:result is None
2023-07-15T07:41:15.760 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2023-07-15T07:41:15.762 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi016
2023-07-15T07:41:24.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com'
2023-07-15T07:41:24.764 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-15T07:41:24.884 ERROR:teuthology.orchestra.connection:Error authenticating with smithi016.front.sepia.ceph.com: Authentication failed.
2023-07-15T07:41:24.884 DEBUG:teuthology.orchestra.run.smithi016:> true
2023-07-15T07:41:24.885 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:43:33.780 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com'
2023-07-15T07:43:33.781 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-15T07:43:33.899 ERROR:teuthology.orchestra.connection:Error authenticating with smithi016.front.sepia.ceph.com: Authentication failed.
2023-07-15T07:43:33.899 DEBUG:teuthology.orchestra.run.smithi016:> true
2023-07-15T07:43:33.900 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:45:45.793 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com'
2023-07-15T07:45:45.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-15T07:45:50.915 ERROR:teuthology.orchestra.connection:Error authenticating with smithi016.front.sepia.ceph.com: Authentication failed.
2023-07-15T07:45:50.916 DEBUG:teuthology.orchestra.run.smithi016:> true
2023-07-15T07:45:50.917 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:47:53.804 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com'
2023-07-15T07:47:53.805 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-15T07:47:58.925 ERROR:teuthology.orchestra.connection:Error authenticating with smithi016.front.sepia.ceph.com: Authentication failed.
2023-07-15T07:47:58.926 DEBUG:teuthology.orchestra.run.smithi016:> true
2023-07-15T07:47:58.927 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-15T07:49:58.816 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds
2023-07-15T07:49:58.818 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/dispatcher/supervisor.py", line 227, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/provision/fog.py", line 299, in _wait_for_ready
    self.remote.run(args=cmd, timeout=600)
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/remote.py", line 521, in run
    raise Exception(f'Cannot connect to remote host {self.shortname}')
Exception: Cannot connect to remote host smithi016
2023-07-15T07:49:58.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-15T07:49:58.820 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-15T07:50:03.912 INFO:teuthology.nuke:Checking targets against current locks
2023-07-15T07:50:18.987 DEBUG:teuthology.nuke:shortname: smithi016
2023-07-15T07:50:18.988 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-07-15T07:50:26.528 DEBUG:teuthology.nuke:shortname: smithi081
2023-07-15T07:50:26.528 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-07-15T07:50:26.532 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-14_23:39:06-rados-wip-yuri5-testing-2023-07-14-0757-distro-default-smithi/7338771', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-15 07:36:32.400120', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLc1R+mgu7vHw9QqMgHLb+SMwaXej30ZKrzcEzzmoYRB1m6V+lyr5HghV57Z6EjpSxSpu1Rr802QD9HBCRuESoP18+/kw0639ORyF6mLhW51WWtMz+Sg7cyx2gCw6FpdKEOwMtwxh0op9RWzewUT3rVC6g7h/sw2A83rEviTfxbyJLtfGxm6UZ8lw4otnhHxZHu3Fcid96fXLbtNyT8Wk3cwzX4ucuiF10lt9lMfKW3YTQmtP8SPfMugMeFMOyUEcjYuoEqVTULfRVOAkBw7K7d2zcbCvx+8lz6a6VBXCIp+WQp7lIQhad88J/5rxmkV70Ca2obq6qyelOkQ/95jdE0WEDXKDcNA2Tg6ElkvP/o+k1WxPYQBiPizDJDkDSvblt8SaFK3hldjemeaEUnM5wtbEeu34EoNAomQJECMalVX9VCAAvCbvLcMW3+5Dp+6b2m7z7v5aeWNfmAE59bPYXddx80N2gy/ISPJvgPvKwSxB/acAB390wTCiDW2ll5Rc='}
2023-07-15T07:50:26.555 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-14_23:39:06-rados-wip-yuri5-testing-2023-07-14-0757-distro-default-smithi/7338744', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-15 07:29:18.233965', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDN5/yEchiLuaiub31xghzB7fXtup0qE1HpLQDNFz0UCdR7+2O/11xAWGoQ4NowabHbJZswBWWk2rrhPeCXMMf213654bVEsMcdKE01LtXjMHvEkvaVMZni/EhEbZB8srEPr9ZK+oPACtRAvCyjsIo2FCXQPfo/OmFe7n5ShCj0RSLjnkwyyVxMbjBWVPNWyLB497RM3jR9M9w7JRGxeo2dmzj/iaPsQjDMadjaAFKnpPZMmNNGt7aeDzqiRJOgzH4YL/NqAvc3Glt56AwPy8bkOmCVA4rno0+k2NxwvfYf4S1X84vFsy/u6n8HJa/H0JUVG82my7hT0j2Ak67YkWNVDjFTUzW7PmEV0cPXytSowG4cuVTOtq1XJgBqgySl8Z1WPWnj4kkUyBLPT6w0Q607dD29i9mz22/8cliFqU4sEzXu+ytC7l/nPIyd/zD7jUamLGqA8JGL9qCuDmXwlJbqvlUPvw6Uqq9Vvv0ZlZm3G4GN0/kBAy1SD0cDXK7zXx8='}
2023-07-15T07:50:26.589 INFO:teuthology.orchestra.console:Power off smithi016
2023-07-15T07:50:26.589 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-15T07:50:26.607 INFO:teuthology.orchestra.console:Power off smithi081
2023-07-15T07:50:26.608 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-15T07:50:26.617 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:50:26.631 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:50:30.644 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:50:30.655 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:50:35.761 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:50:35.769 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:50:40.873 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:50:40.885 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:50:41.099 INFO:teuthology.orchestra.console:Power off for smithi081 completed
2023-07-15T07:50:51.236 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:50:51.271 INFO:teuthology.lock.ops:unlocked: smithi081.front.sepia.ceph.com
2023-07-15T07:50:56.350 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-15T07:50:56.561 INFO:teuthology.orchestra.console:Power off for smithi016 completed
2023-07-15T07:51:02.132 INFO:teuthology.lock.ops:unlocked: smithi016.front.sepia.ceph.com
2023-07-15T07:51:02.369 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/dispatcher/supervisor.py", line 57, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/dispatcher/supervisor.py", line 227, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/provision/fog.py", line 299, in _wait_for_ready
    self.remote.run(args=cmd, timeout=600)
  File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/orchestra/remote.py", line 521, in run
    raise Exception(f'Cannot connect to remote host {self.shortname}')
Exception: Cannot connect to remote host smithi016