2023-07-21T21:58:00.203 INFO:root:teuthology version: 0.0.1.dev153+g79491cb 2023-07-21T21:58:00.205 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T21:58:00.205 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T21:58:00.424 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa10acbe8b0>, signals=[15]) 2023-07-21T21:58:00.426 INFO:teuthology.lock.ops:Start node 'smithi078.front.sepia.ceph.com' reimaging 2023-07-21T21:58:00.427 INFO:teuthology.lock.ops:Updating [smithi078.front.sepia.ceph.com]: reset os type and version on server 2023-07-21T21:58:00.428 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2023-07-21T21:58:00.452 INFO:teuthology.lock.ops:Start node 'smithi121.front.sepia.ceph.com' reimaging 2023-07-21T21:58:00.453 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: reset os type and version on server 2023-07-21T21:58:00.453 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2023-07-21T21:58:00.454 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /host {"name": "smithi078"} 2023-07-21T21:58:00.477 INFO:teuthology.lock.ops:Start node 'smithi159.front.sepia.ceph.com' reimaging 2023-07-21T21:58:00.478 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: reset os type and version on server 2023-07-21T21:58:00.478 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2023-07-21T21:58:00.481 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /host {"name": "smithi121"} 2023-07-21T21:58:00.502 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /host {"name": "smithi159"} 2023-07-21T21:58:00.548 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-07-21T21:58:00.554 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-07-21T21:58:00.563 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-07-21T21:58:00.598 DEBUG:teuthology.provision.fog.smithi121:Requesting image smithi_rhel_8.6 (ID 5) 2023-07-21T21:58:00.598 DEBUG:teuthology.provision.fog.smithi121:FOG request: PUT /host/226 {"imageID": 5} 2023-07-21T21:58:00.602 DEBUG:teuthology.provision.fog.smithi159:Requesting image smithi_rhel_8.6 (ID 5) 2023-07-21T21:58:00.603 DEBUG:teuthology.provision.fog.smithi159:FOG request: PUT /host/264 {"imageID": 5} 2023-07-21T21:58:00.606 DEBUG:teuthology.provision.fog.smithi078:Requesting image smithi_rhel_8.6 (ID 5) 2023-07-21T21:58:00.607 DEBUG:teuthology.provision.fog.smithi078:FOG request: PUT /host/183 {"imageID": 5} 2023-07-21T21:58:00.649 INFO:teuthology.provision.fog.smithi078:Scheduling deploy of rhel 8.6 2023-07-21T21:58:00.649 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-07-21T21:58:00.652 INFO:teuthology.provision.fog.smithi159:Scheduling deploy of rhel 8.6 2023-07-21T21:58:00.652 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None 2023-07-21T21:58:00.653 INFO:teuthology.provision.fog.smithi121:Scheduling deploy of rhel 8.6 2023-07-21T21:58:00.653 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None 2023-07-21T21:58:00.695 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /tasktype {"name": "deploy"} 2023-07-21T21:58:00.702 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /tasktype {"name": "deploy"} 2023-07-21T21:58:00.707 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /tasktype {"name": "deploy"} 2023-07-21T21:58:00.733 DEBUG:teuthology.provision.fog.smithi159:FOG request: POST /host/264/task {"taskTypeID": 1} 2023-07-21T21:58:00.736 DEBUG:teuthology.provision.fog.smithi121:FOG request: POST /host/226/task {"taskTypeID": 1} 2023-07-21T21:58:00.745 DEBUG:teuthology.provision.fog.smithi078:FOG request: POST /host/183/task {"taskTypeID": 1} 2023-07-21T21:58:00.885 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None 2023-07-21T21:58:00.888 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None 2023-07-21T21:58:00.892 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-07-21T21:58:00.986 INFO:teuthology.orchestra.console:Power off smithi159 2023-07-21T21:58:00.987 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T21:58:01.005 INFO:teuthology.orchestra.console:Power off smithi121 2023-07-21T21:58:01.006 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T21:58:01.021 INFO:teuthology.orchestra.console:Power off smithi078 2023-07-21T21:58:01.022 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T21:58:01.032 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:01.042 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:01.051 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:05.064 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:05.075 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:05.087 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:10.179 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:10.189 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:10.199 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:15.292 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:15.300 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:15.311 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:15.521 INFO:teuthology.orchestra.console:Power off for smithi078 completed 2023-07-21T21:58:15.622 INFO:teuthology.orchestra.console:Power on smithi078 2023-07-21T21:58:15.623 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-21T21:58:15.648 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:20.404 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:20.413 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:20.657 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:20.793 INFO:teuthology.orchestra.console:Power off for smithi159 completed 2023-07-21T21:58:20.894 INFO:teuthology.orchestra.console:Power on smithi159 2023-07-21T21:58:20.894 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-21T21:58:25.517 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:25.771 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:25.873 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:30.632 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:30.885 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:30.893 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:34.908 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:35.996 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:36.005 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:40.020 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:41.108 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:41.116 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:41.325 INFO:teuthology.orchestra.console:Power on for smithi078 completed 2023-07-21T21:58:41.426 INFO:teuthology.provision.fog.smithi078:Waiting for deploy to finish 2023-07-21T21:58:41.427 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-07-21T21:58:46.435 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:46.545 INFO:teuthology.orchestra.console:Power on for smithi159 completed 2023-07-21T21:58:46.646 INFO:teuthology.provision.fog.smithi159:Waiting for deploy to finish 2023-07-21T21:58:46.703 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None 2023-07-21T21:58:51.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:58:56.521 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:58:59.600 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2023-07-21T21:59:01.508 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-07-21T21:59:06.517 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:59:06.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:59:19.600 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2023-07-21T21:59:20.538 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:59:25.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:59:30.609 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:59:31.587 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-07-21T21:59:36.593 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2023-07-21T21:59:40.596 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:59:40.725 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds 2023-07-21T21:59:40.826 ERROR:teuthology.orchestra.console:Failed to power off smithi121 2023-07-21T21:59:40.926 INFO:teuthology.orchestra.console:Power on smithi121 2023-07-21T21:59:40.927 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-21T21:59:40.947 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T21:59:41.069 INFO:teuthology.orchestra.console:Power on for smithi121 completed 2023-07-21T21:59:41.171 INFO:teuthology.provision.fog.smithi121:Waiting for deploy to finish 2023-07-21T21:59:41.171 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None 2023-07-21T21:59:41.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:59:42.596 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:59:45.747 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-07-21T21:59:51.673 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-07-21T21:59:51.749 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:59:54.800 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-07-21T22:00:00.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:00:04.208 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2023-07-21T22:00:06.748 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-07-21T22:00:11.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:00:16.759 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-07-21T22:00:16.874 ERROR:teuthology.orchestra.connection:Error authenticating with smithi159.front.sepia.ceph.com: Authentication failed. 2023-07-21T22:00:16.876 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-21T22:00:16.877 DEBUG:teuthology.orchestra.run.smithi159:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-21T22:00:16.878 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:17.882 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:18.888 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:19.894 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:20.899 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:21.905 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:22.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:00:22.910 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:23.915 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:24.434 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-07-21T22:00:24.921 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:25.926 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:26.931 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:27.937 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:28.942 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:29.948 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:30.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:00:35.441 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-07-21T22:00:40.443 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:40.511 ERROR:teuthology.orchestra.connection:Error authenticating with smithi121.front.sepia.ceph.com: Authentication failed. 2023-07-21T22:00:40.512 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-21T22:00:40.512 DEBUG:teuthology.orchestra.run.smithi121:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-21T22:00:40.513 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:40.544 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:41.520 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:41.552 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:42.527 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:42.560 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:43.536 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:43.568 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:44.544 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:44.576 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:45.554 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:45.586 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:46.563 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:46.595 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:47.571 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:47.603 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:48.580 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:48.611 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:49.589 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:49.620 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:50.598 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:50.630 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:51.608 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:51.639 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:52.617 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:52.648 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:53.626 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:53.657 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:54.634 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:54.666 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:55.508 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-07-21T22:00:56.905 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:56.906 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:57.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:00:57.912 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:57.913 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:58.919 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:58.920 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:59.925 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:00:59.926 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:00.112 WARNING:teuthology.provision.fog.smithi078:[Errno None] Unable to connect to port 22 on 172.21.15.78 2023-07-21T22:01:00.931 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:00.932 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:01.938 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:01.938 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:02.944 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:02.944 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:03.951 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:03.951 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:04.957 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:04.957 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:05.963 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:05.963 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:06.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:01:11.121 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:11.121 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:12.126 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:12.126 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:13.135 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:13.136 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:14.142 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:14.143 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:14.192 WARNING:teuthology.provision.fog.smithi078:[Errno None] Unable to connect to port 22 on 172.21.15.78 2023-07-21T22:01:15.148 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:15.149 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:16.154 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:16.155 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:17.161 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:17.162 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:18.167 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:18.168 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:19.174 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:19.174 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:20.181 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:20.182 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:20.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:01:21.187 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:21.187 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:22.193 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:22.194 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:23.199 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:23.200 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:23.250 WARNING:teuthology.provision.fog.smithi078:[Errno None] Unable to connect to port 22 on 172.21.15.78 2023-07-21T22:01:24.206 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:24.207 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:25.213 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:25.213 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:26.219 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:26.220 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:27.226 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:27.226 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:28.231 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:28.232 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:29.237 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:29.238 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:29.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:01:30.243 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:30.244 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:31.250 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:31.250 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:32.256 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:32.256 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:32.308 WARNING:teuthology.provision.fog.smithi078:[Errno None] Unable to connect to port 22 on 172.21.15.78 2023-07-21T22:01:33.263 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:33.263 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:34.269 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:34.270 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:35.275 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:35.276 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:36.281 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:36.282 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:37.287 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:37.288 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:38.293 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:38.294 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:38.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:01:39.299 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:39.300 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:40.306 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:40.306 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:41.311 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:41.312 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:41.360 WARNING:teuthology.provision.fog.smithi078:[Errno None] Unable to connect to port 22 on 172.21.15.78 2023-07-21T22:01:42.317 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:42.318 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:43.323 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:43.323 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:44.328 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:44.329 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:45.335 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:46.341 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:47.348 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:47.361 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:01:48.355 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:49.362 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:50.370 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:50.416 WARNING:teuthology.provision.fog.smithi078:[Errno None] Unable to connect to port 22 on 172.21.15.78 2023-07-21T22:01:51.378 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:52.384 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:53.391 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:54.398 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:55.404 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-21T22:01:56.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:02:56.420 WARNING:teuthology.provision.fog.smithi078:timed out 2023-07-21T22:03:02.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:03:02.756 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-21T22:03:02.757 DEBUG:teuthology.orchestra.run.smithi078:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-21T22:03:03.095 INFO:teuthology.provision.fog.smithi078:Node is ready 2023-07-21T22:03:03.095 DEBUG:teuthology.orchestra.run.smithi078:> hostname 2023-07-21T22:03:03.116 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078.front.sepia.ceph.com 2023-07-21T22:03:03.117 DEBUG:teuthology.orchestra.run.smithi078:> grep smithi078.front.sepia.ceph.com /etc/hosts 2023-07-21T22:03:03.179 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78 smithi078.front.sepia.ceph.com smithi078 2023-07-21T22:03:03.179 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname smithi078 2023-07-21T22:03:03.301 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i -e 's/smithi078.front.sepia.ceph.com/smithi078/g' /etc/hosts 2023-07-21T22:03:03.336 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i -e 's/172.21.15.78/172.21.15.78/g' /etc/hosts 2023-07-21T22:03:03.409 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i -e 's/smithi078.front.sepia.ceph.com/smithi078/g' /etc/hostname 2023-07-21T22:03:03.482 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname smithi078 2023-07-21T22:03:03.554 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2023-07-21T22:03:03.614 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T22:03:03.615 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T22:03:03.615 INFO:teuthology.orchestra.run.smithi078.stdout:ID="rhel" 2023-07-21T22:03:03.615 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="fedora" 2023-07-21T22:03:03.615 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="8.6" 2023-07-21T22:03:03.615 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el8" 2023-07-21T22:03:03.616 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T22:03:03.616 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2023-07-21T22:03:03.616 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T22:03:03.616 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T22:03:03.616 INFO:teuthology.orchestra.run.smithi078.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T22:03:03.616 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T22:03:03.617 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-07-21T22:03:03.617 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T22:03:03.617 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T22:03:03.617 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T22:03:03.617 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T22:03:03.618 INFO:teuthology.provision.fog.smithi078:Deploy complete! 2023-07-21T22:03:03.619 DEBUG:teuthology.parallel:result is None 2023-07-21T22:03:32.336 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-07-21T22:03:32.337 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi159 2023-07-21T22:03:41.338 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2023-07-21T22:03:41.339 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:03:41.341 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.159 2023-07-21T22:03:44.624 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-07-21T22:03:44.626 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi121 2023-07-21T22:03:50.345 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2023-07-21T22:03:50.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:03:55.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-07-21T22:03:55.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:04:00.358 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-07-21T22:04:00.558 DEBUG:teuthology.orchestra.run.smithi159:> true 2023-07-21T22:04:00.977 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2023-07-21T22:04:00.977 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-21T22:04:00.977 DEBUG:teuthology.orchestra.run.smithi159:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-21T22:04:01.027 INFO:teuthology.provision.fog.smithi159:Node is ready 2023-07-21T22:04:01.027 DEBUG:teuthology.orchestra.run.smithi159:> hostname 2023-07-21T22:04:01.080 INFO:teuthology.orchestra.run.smithi159.stdout:smithi159.front.sepia.ceph.com 2023-07-21T22:04:01.080 DEBUG:teuthology.orchestra.run.smithi159:> grep smithi159.front.sepia.ceph.com /etc/hosts 2023-07-21T22:04:01.128 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159 smithi159.front.sepia.ceph.com smithi159 2023-07-21T22:04:01.129 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159 2023-07-21T22:04:01.246 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi159.front.sepia.ceph.com/smithi159/g' /etc/hosts 2023-07-21T22:04:01.260 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/172.21.15.159/172.21.15.159/g' /etc/hosts 2023-07-21T22:04:01.321 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi159.front.sepia.ceph.com/smithi159/g' /etc/hostname 2023-07-21T22:04:01.375 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159 2023-07-21T22:04:01.431 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2023-07-21T22:04:01.480 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-07-21T22:04:01.481 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="Ubuntu" 2023-07-21T22:04:01.481 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="22.04" 2023-07-21T22:04:01.481 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-07-21T22:04:01.482 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_CODENAME=jammy 2023-07-21T22:04:01.482 INFO:teuthology.orchestra.run.smithi159.stdout:ID=ubuntu 2023-07-21T22:04:01.482 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE=debian 2023-07-21T22:04:01.482 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://www.ubuntu.com/" 2023-07-21T22:04:01.483 INFO:teuthology.orchestra.run.smithi159.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-07-21T22:04:01.483 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-07-21T22:04:01.483 INFO:teuthology.orchestra.run.smithi159.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-07-21T22:04:01.483 INFO:teuthology.orchestra.run.smithi159.stdout:UBUNTU_CODENAME=jammy 2023-07-21T22:04:01.485 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi159's OS to be rhel 8.6 but found ubuntu 22.04 2023-07-21T22:04:01.485 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T22:04:01.486 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T22:04:05.834 INFO:teuthology.nuke:Checking targets against current locks 2023-07-21T22:04:05.889 DEBUG:teuthology.nuke:shortname: smithi078 2023-07-21T22:04:05.889 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T22:04:05.912 DEBUG:teuthology.nuke:shortname: smithi121 2023-07-21T22:04:05.912 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T22:04:05.913 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347734', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:57:59.313782', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-21T22:04:05.944 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347735', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:57:59.761058', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOLuu8GtmxzCe4rJCs8HnqxfUkKA0te4T07aB5fNQ3gFlx3xB4s0ragw673/s1JN40qR5PC7mvyr5Yf/sXEGLh8='} 2023-07-21T22:04:05.946 DEBUG:teuthology.nuke:shortname: smithi159 2023-07-21T22:04:05.947 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T22:04:05.984 INFO:teuthology.orchestra.console:Power off smithi078 2023-07-21T22:04:05.985 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T22:04:06.002 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347735', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:57:59.763434', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKyoqPSL61CZqOLNk/GSUIIKsSY7Ji+ihWOUq1et61+kzdWcxzjLH/yO7uBasD0XTXnctYhxkOZMVD7T8CwNAHA='} 2023-07-21T22:04:06.009 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:06.040 INFO:teuthology.orchestra.console:Power off smithi121 2023-07-21T22:04:06.041 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T22:04:06.068 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:06.087 INFO:teuthology.orchestra.console:Power off smithi159 2023-07-21T22:04:06.089 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T22:04:06.118 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:09.360 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-07-21T22:04:09.361 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:04:09.451 DEBUG:teuthology.orchestra.run.smithi121:> true 2023-07-21T22:04:10.053 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:10.103 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:10.143 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:10.921 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-07-21T22:04:10.922 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-21T22:04:10.922 DEBUG:teuthology.orchestra.run.smithi121:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-21T22:04:10.970 INFO:teuthology.provision.fog.smithi121:Node is ready 2023-07-21T22:04:10.971 DEBUG:teuthology.orchestra.run.smithi121:> hostname 2023-07-21T22:04:11.018 INFO:teuthology.orchestra.run.smithi121.stdout:smithi121 2023-07-21T22:04:11.019 DEBUG:teuthology.orchestra.run.smithi121:> grep smithi121 /etc/hosts 2023-07-21T22:04:11.067 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.15.121 smithi121 smithi121 2023-07-21T22:04:11.068 DEBUG:teuthology.orchestra.run.smithi121:> sudo hostname smithi121 2023-07-21T22:04:11.123 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i -e 's/smithi121/smithi121/g' /etc/hosts 2023-07-21T22:04:11.179 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i -e 's/172.21.15.121/172.21.15.121/g' /etc/hosts 2023-07-21T22:04:11.236 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i -e 's/smithi121/smithi121/g' /etc/hostname 2023-07-21T22:04:11.293 DEBUG:teuthology.orchestra.run.smithi121:> sudo hostname smithi121 2023-07-21T22:04:11.346 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2023-07-21T22:04:11.394 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-07-21T22:04:11.395 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="Ubuntu" 2023-07-21T22:04:11.395 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="22.04" 2023-07-21T22:04:11.395 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-07-21T22:04:11.395 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_CODENAME=jammy 2023-07-21T22:04:11.395 INFO:teuthology.orchestra.run.smithi121.stdout:ID=ubuntu 2023-07-21T22:04:11.395 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE=debian 2023-07-21T22:04:11.396 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://www.ubuntu.com/" 2023-07-21T22:04:11.396 INFO:teuthology.orchestra.run.smithi121.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-07-21T22:04:11.396 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-07-21T22:04:11.396 INFO:teuthology.orchestra.run.smithi121.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-07-21T22:04:11.396 INFO:teuthology.orchestra.run.smithi121.stdout:UBUNTU_CODENAME=jammy 2023-07-21T22:04:14.168 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:14.214 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:14.254 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:19.280 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:19.326 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:19.366 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:24.392 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:24.436 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:24.477 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:24.691 INFO:teuthology.orchestra.console:Power off for smithi159 completed 2023-07-21T22:04:24.855 INFO:teuthology.lock.ops:unlocked: smithi159.front.sepia.ceph.com 2023-07-21T22:04:29.504 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:29.550 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:29.763 INFO:teuthology.orchestra.console:Power off for smithi121 completed 2023-07-21T22:04:35.768 INFO:teuthology.lock.ops:unlocked: smithi121.front.sepia.ceph.com 2023-07-21T22:04:38.876 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:43.988 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:49.104 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:54.219 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:04:54.429 INFO:teuthology.orchestra.console:Power off for smithi078 completed 2023-07-21T22:05:00.848 INFO:teuthology.lock.ops:unlocked: smithi078.front.sepia.ceph.com 2023-07-21T22:05:06.242 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/dispatcher/supervisor.py", line 57, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi159's OS to be rhel 8.6 but found ubuntu 22.04