2023-08-27T16:14:34.252 INFO:root:teuthology version: 0.0.1.dev185+g449a1bc
2023-08-27T16:14:34.252 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-27T16:14:34.429 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f4804f0b880>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f4804c10a60>, signals=[15])
2023-08-27T16:14:34.429 INFO:teuthology.lock.ops:Start node 'smithi123.front.sepia.ceph.com' reimaging
2023-08-27T16:14:34.430 INFO:teuthology.lock.ops:Updating [smithi123.front.sepia.ceph.com]: reset os type and version on server
2023-08-27T16:14:34.430 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server
2023-08-27T16:14:34.458 INFO:teuthology.lock.ops:Start node 'smithi149.front.sepia.ceph.com' reimaging
2023-08-27T16:14:34.458 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: reset os type and version on server
2023-08-27T16:14:34.458 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server
2023-08-27T16:14:34.460 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /host {"name": "smithi123"}
2023-08-27T16:14:34.481 INFO:teuthology.lock.ops:Start node 'smithi167.front.sepia.ceph.com' reimaging
2023-08-27T16:14:34.482 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: reset os type and version on server
2023-08-27T16:14:34.482 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server
2023-08-27T16:14:34.483 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /host {"name": "smithi149"}
2023-08-27T16:14:34.506 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /host {"name": "smithi167"}
2023-08-27T16:14:34.531 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-08-27T16:14:34.550 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-08-27T16:14:34.565 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-08-27T16:14:34.569 DEBUG:teuthology.provision.fog.smithi123:Requesting image smithi_rhel_8.6 (ID 5)
2023-08-27T16:14:34.569 DEBUG:teuthology.provision.fog.smithi123:FOG request: PUT /host/228 {"imageID": 5}
2023-08-27T16:14:34.595 DEBUG:teuthology.provision.fog.smithi149:Requesting image smithi_rhel_8.6 (ID 5)
2023-08-27T16:14:34.595 DEBUG:teuthology.provision.fog.smithi149:FOG request: PUT /host/254 {"imageID": 5}
2023-08-27T16:14:34.598 DEBUG:teuthology.provision.fog.smithi167:Requesting image smithi_rhel_8.6 (ID 5)
2023-08-27T16:14:34.598 DEBUG:teuthology.provision.fog.smithi167:FOG request: PUT /host/272 {"imageID": 5}
2023-08-27T16:14:34.608 INFO:teuthology.provision.fog.smithi123:Scheduling deploy of rhel 8.6
2023-08-27T16:14:34.609 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None
2023-08-27T16:14:34.637 INFO:teuthology.provision.fog.smithi149:Scheduling deploy of rhel 8.6
2023-08-27T16:14:34.637 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-08-27T16:14:34.640 INFO:teuthology.provision.fog.smithi167:Scheduling deploy of rhel 8.6
2023-08-27T16:14:34.640 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /task/active None
2023-08-27T16:14:34.667 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /tasktype {"name": "deploy"}
2023-08-27T16:14:34.706 DEBUG:teuthology.provision.fog.smithi123:FOG request: POST /host/228/task {"taskTypeID": 1}
2023-08-27T16:14:34.715 DEBUG:teuthology.provision.fog.smithi149:Canceling deploy task with ID 465962
2023-08-27T16:14:34.715 DEBUG:teuthology.provision.fog.smithi149:FOG request: DELETE /task/cancel {"id": 465962}
2023-08-27T16:14:34.717 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /tasktype {"name": "deploy"}
2023-08-27T16:14:34.758 DEBUG:teuthology.provision.fog.smithi167:FOG request: POST /host/272/task {"taskTypeID": 1}
2023-08-27T16:14:34.764 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /tasktype {"name": "deploy"}
2023-08-27T16:14:34.768 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None
2023-08-27T16:14:34.796 DEBUG:teuthology.provision.fog.smithi149:FOG request: POST /host/254/task {"taskTypeID": 1}
2023-08-27T16:14:34.813 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /task/active None
2023-08-27T16:14:34.843 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-08-27T16:14:34.858 INFO:teuthology.orchestra.console.smithi123:Power off
2023-08-27T16:14:34.859 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-27T16:14:34.880 DEBUG:teuthology.orchestra.console.smithi123:power off output: Chassis Power Control: Down/Off
2023-08-27T16:14:34.880 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-27T16:14:34.898 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off
2023-08-27T16:14:34.909 INFO:teuthology.orchestra.console.smithi167:Power off
2023-08-27T16:14:34.909 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-27T16:14:34.926 INFO:teuthology.orchestra.console.smithi149:Power off
2023-08-27T16:14:34.926 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-27T16:14:34.933 DEBUG:teuthology.orchestra.console.smithi167:power off output: Chassis Power Control: Down/Off
2023-08-27T16:14:34.933 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-27T16:14:34.944 DEBUG:teuthology.orchestra.console.smithi149:power off output: Chassis Power Control: Down/Off
2023-08-27T16:14:34.945 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-27T16:14:34.952 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off
2023-08-27T16:14:34.962 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off
2023-08-27T16:14:35.000 INFO:teuthology.orchestra.console.smithi123:Power off completed
2023-08-27T16:14:35.053 INFO:teuthology.orchestra.console.smithi167:Power off completed
2023-08-27T16:14:35.063 INFO:teuthology.orchestra.console.smithi149:Power off completed
2023-08-27T16:14:35.100 INFO:teuthology.orchestra.console.smithi123:Power on
2023-08-27T16:14:35.100 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-08-27T16:14:35.119 DEBUG:teuthology.orchestra.console.smithi123:power on output: Chassis Power Control: Up/On
2023-08-27T16:14:35.119 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-27T16:14:35.141 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off
2023-08-27T16:14:35.154 INFO:teuthology.orchestra.console.smithi167:Power on
2023-08-27T16:14:35.154 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-08-27T16:14:35.165 INFO:teuthology.orchestra.console.smithi149:Power on
2023-08-27T16:14:35.166 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-08-27T16:14:35.175 DEBUG:teuthology.orchestra.console.smithi167:power on output: Chassis Power Control: Up/On
2023-08-27T16:14:35.176 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-27T16:14:35.187 DEBUG:teuthology.orchestra.console.smithi149:power on output: Chassis Power Control: Up/On
2023-08-27T16:14:35.187 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-27T16:14:35.194 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off
2023-08-27T16:14:35.206 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off
2023-08-27T16:14:39.146 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-27T16:14:39.195 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-27T16:14:39.207 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-27T16:14:39.254 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on
2023-08-27T16:14:39.304 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on
2023-08-27T16:14:39.315 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on
2023-08-27T16:14:39.355 INFO:teuthology.orchestra.console.smithi123:Power on completed
2023-08-27T16:14:39.406 INFO:teuthology.orchestra.console.smithi167:Power on completed
2023-08-27T16:14:39.416 INFO:teuthology.orchestra.console.smithi149:Power on completed
2023-08-27T16:14:39.457 INFO:teuthology.provision.fog.smithi123:Waiting for deploy to finish
2023-08-27T16:14:39.457 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None
2023-08-27T16:14:39.506 INFO:teuthology.provision.fog.smithi167:Waiting for deploy to finish
2023-08-27T16:14:39.507 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /task/active None
2023-08-27T16:14:39.517 INFO:teuthology.provision.fog.smithi149:Waiting for deploy to finish
2023-08-27T16:14:39.517 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-08-27T16:14:54.535 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None
2023-08-27T16:14:54.588 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /task/active None
2023-08-27T16:14:54.605 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-08-27T16:15:09.614 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None
2023-08-27T16:15:09.664 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /task/active None
2023-08-27T16:15:09.668 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-08-27T16:15:24.691 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None
2023-08-27T16:15:24.747 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /task/active None
2023-08-27T16:15:24.754 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-08-27T16:15:39.775 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None
2023-08-27T16:15:39.956 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /task/active None
2023-08-27T16:15:40.102 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-08-27T16:15:55.179 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-08-27T16:15:55.180 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /task/active None
2023-08-27T16:15:55.181 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None
2023-08-27T16:16:10.267 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /task/active None
2023-08-27T16:16:10.269 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None
2023-08-27T16:16:10.270 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-08-27T16:16:25.359 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /task/active None
2023-08-27T16:16:25.360 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None
2023-08-27T16:16:25.361 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-08-27T16:16:40.463 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-08-27T16:16:40.465 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None
2023-08-27T16:16:40.466 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /task/active None
2023-08-27T16:16:40.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:16:55.568 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-08-27T16:16:55.570 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /task/active None
2023-08-27T16:16:58.987 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123
2023-08-27T16:17:04.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:17:08.043 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123
2023-08-27T16:17:10.663 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-08-27T16:17:10.665 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /task/active None
2023-08-27T16:17:14.047 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:17:17.103 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123
2023-08-27T16:17:23.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:17:25.734 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-08-27T16:17:25.736 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /task/active None
2023-08-27T16:17:25.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:17:26.187 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123
2023-08-27T16:17:32.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:17:35.247 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123
2023-08-27T16:17:40.807 DEBUG:teuthology.provision.fog.smithi167:FOG request: GET /task/active None
2023-08-27T16:17:40.874 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:17:41.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:17:44.299 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2023-08-27T16:17:50.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:17:53.391 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2023-08-27T16:17:59.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:17:59.403 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-08-27T16:17:59.659 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123
2023-08-27T16:18:02.475 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2023-08-27T16:18:05.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:18:05.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:18:08.463 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-08-27T16:18:08.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:18:08.715 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123
2023-08-27T16:18:11.535 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2023-08-27T16:18:14.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:18:14.716 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:18:14.853 ERROR:teuthology.orchestra.connection:Error authenticating with smithi123.front.sepia.ceph.com: Authentication failed.
2023-08-27T16:18:14.854 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-08-27T16:18:14.854 DEBUG:teuthology.orchestra.run.smithi123:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-08-27T16:18:14.855 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:15.860 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:16.866 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:17.515 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-08-27T16:18:17.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:18:17.871 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:18.876 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:19.882 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:20.587 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2023-08-27T16:18:20.887 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:21.892 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:22.897 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:23.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:18:23.903 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:24.908 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:25.914 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:26.571 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-08-27T16:18:26.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:18:26.919 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:27.924 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:28.930 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:29.935 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:30.940 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:31.946 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:32.572 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:18:32.952 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:33.958 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:34.963 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:35.631 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-08-27T16:18:35.968 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:36.974 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:37.979 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:38.985 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:39.990 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:40.996 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:41.633 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:18:42.002 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:43.007 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:44.014 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:45.020 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:46.025 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:47.031 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:48.036 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:49.042 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:50.047 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:51.053 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:52.058 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:53.063 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:54.069 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:55.074 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:56.079 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:57.084 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:58.090 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:59.095 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:18:59.630 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-08-27T16:18:59.630 DEBUG:teuthology.orchestra.run.smithi149:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-08-27T16:18:59.886 INFO:teuthology.provision.fog.smithi149:Node is ready
2023-08-27T16:18:59.886 DEBUG:teuthology.orchestra.run.smithi149:> hostname
2023-08-27T16:18:59.901 INFO:teuthology.orchestra.run.smithi149.stdout:smithi022
2023-08-27T16:18:59.901 DEBUG:teuthology.orchestra.run.smithi149:> grep smithi022 /etc/hosts
2023-08-27T16:18:59.958 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022
2023-08-27T16:18:59.958 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname smithi149
2023-08-27T16:19:00.066 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/smithi022/smithi149/g' /etc/hosts
2023-08-27T16:19:00.092 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/172.21.15.22/172.21.15.149/g' /etc/hosts
2023-08-27T16:19:00.102 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:19:00.159 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/smithi022/smithi149/g' /etc/hostname
2023-08-27T16:19:00.226 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname smithi149
2023-08-27T16:19:00.292 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release
2023-08-27T16:19:00.348 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="Red Hat Enterprise Linux"
2023-08-27T16:19:00.349 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="8.6 (Ootpa)"
2023-08-27T16:19:00.349 INFO:teuthology.orchestra.run.smithi149.stdout:ID="rhel"
2023-08-27T16:19:00.349 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="fedora"
2023-08-27T16:19:00.349 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="8.6"
2023-08-27T16:19:00.349 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el8"
2023-08-27T16:19:00.349 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-08-27T16:19:00.349 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31"
2023-08-27T16:19:00.349 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-08-27T16:19:00.350 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://www.redhat.com/"
2023-08-27T16:19:00.350 INFO:teuthology.orchestra.run.smithi149.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-08-27T16:19:00.350 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-08-27T16:19:00.350 INFO:teuthology.orchestra.run.smithi149.stdout:
2023-08-27T16:19:00.350 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-08-27T16:19:00.350 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-08-27T16:19:00.350 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-08-27T16:19:00.350 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-08-27T16:19:00.351 INFO:teuthology.provision.fog.smithi149:Deploy complete!
2023-08-27T16:19:00.352 DEBUG:teuthology.parallel:result is None
2023-08-27T16:19:01.109 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:19:02.115 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:19:03.120 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:19:04.125 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:19:05.130 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:19:06.136 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:19:07.141 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:19:08.147 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:19:09.152 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:19:10.157 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T16:19:41.635 WARNING:teuthology.provision.fog.smithi167:timed out
2023-08-27T16:19:47.637 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:19:47.969 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-08-27T16:19:47.970 DEBUG:teuthology.orchestra.run.smithi167:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-08-27T16:19:48.264 INFO:teuthology.provision.fog.smithi167:Node is ready
2023-08-27T16:19:48.264 DEBUG:teuthology.orchestra.run.smithi167:> hostname
2023-08-27T16:19:48.282 INFO:teuthology.orchestra.run.smithi167.stdout:smithi167.front.sepia.ceph.com
2023-08-27T16:19:48.283 DEBUG:teuthology.orchestra.run.smithi167:> grep smithi167.front.sepia.ceph.com /etc/hosts
2023-08-27T16:19:48.340 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.15.167 smithi167.front.sepia.ceph.com smithi167
2023-08-27T16:19:48.341 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname smithi167
2023-08-27T16:19:48.458 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/smithi167.front.sepia.ceph.com/smithi167/g' /etc/hosts
2023-08-27T16:19:48.486 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/172.21.15.167/172.21.15.167/g' /etc/hosts
2023-08-27T16:19:48.554 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/smithi167.front.sepia.ceph.com/smithi167/g' /etc/hostname
2023-08-27T16:19:48.621 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname smithi167
2023-08-27T16:19:48.687 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release
2023-08-27T16:19:48.743 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="Red Hat Enterprise Linux"
2023-08-27T16:19:48.743 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="8.6 (Ootpa)"
2023-08-27T16:19:48.743 INFO:teuthology.orchestra.run.smithi167.stdout:ID="rhel"
2023-08-27T16:19:48.744 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="fedora"
2023-08-27T16:19:48.744 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="8.6"
2023-08-27T16:19:48.744 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el8"
2023-08-27T16:19:48.744 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-08-27T16:19:48.744 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31"
2023-08-27T16:19:48.744 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-08-27T16:19:48.744 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://www.redhat.com/"
2023-08-27T16:19:48.744 INFO:teuthology.orchestra.run.smithi167.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-08-27T16:19:48.744 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-08-27T16:19:48.744 INFO:teuthology.orchestra.run.smithi167.stdout:
2023-08-27T16:19:48.744 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-08-27T16:19:48.744 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-08-27T16:19:48.744 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-08-27T16:19:48.745 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-08-27T16:19:48.745 INFO:teuthology.provision.fog.smithi167:Deploy complete!
2023-08-27T16:19:48.745 DEBUG:teuthology.parallel:result is None
2023-08-27T16:21:25.739 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2023-08-27T16:21:25.740 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi123
2023-08-27T16:21:34.747 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com'
2023-08-27T16:21:34.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:21:34.813 DEBUG:teuthology.orchestra.run.smithi123:> true
2023-08-27T16:21:34.879 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com'
2023-08-27T16:21:34.879 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-08-27T16:21:34.879 DEBUG:teuthology.orchestra.run.smithi123:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-08-27T16:21:34.938 INFO:teuthology.provision.fog.smithi123:Node is ready
2023-08-27T16:21:34.939 DEBUG:teuthology.orchestra.run.smithi123:> hostname
2023-08-27T16:21:34.995 INFO:teuthology.orchestra.run.smithi123.stdout:smithi123
2023-08-27T16:21:34.996 DEBUG:teuthology.orchestra.run.smithi123:> grep smithi123 /etc/hosts
2023-08-27T16:21:35.052 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123 smithi123 smithi123
2023-08-27T16:21:35.052 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname smithi123
2023-08-27T16:21:35.119 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i -e 's/smithi123/smithi123/g' /etc/hosts
2023-08-27T16:21:35.186 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i -e 's/172.21.15.123/172.21.15.123/g' /etc/hosts
2023-08-27T16:21:35.253 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i -e 's/smithi123/smithi123/g' /etc/hostname
2023-08-27T16:21:35.319 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname smithi123
2023-08-27T16:21:35.385 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release
2023-08-27T16:21:35.441 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="Red Hat Enterprise Linux"
2023-08-27T16:21:35.442 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8.6 (Ootpa)"
2023-08-27T16:21:35.442 INFO:teuthology.orchestra.run.smithi123.stdout:ID="rhel"
2023-08-27T16:21:35.442 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="fedora"
2023-08-27T16:21:35.442 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8.6"
2023-08-27T16:21:35.442 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8"
2023-08-27T16:21:35.442 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-08-27T16:21:35.442 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31"
2023-08-27T16:21:35.442 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-08-27T16:21:35.442 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://www.redhat.com/"
2023-08-27T16:21:35.442 INFO:teuthology.orchestra.run.smithi123.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-08-27T16:21:35.442 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-08-27T16:21:35.443 INFO:teuthology.orchestra.run.smithi123.stdout:
2023-08-27T16:21:35.443 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-08-27T16:21:35.443 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-08-27T16:21:35.443 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-08-27T16:21:35.443 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-08-27T16:21:35.443 INFO:teuthology.provision.fog.smithi123:Deploy complete!
2023-08-27T16:21:35.443 DEBUG:teuthology.parallel:result is None
2023-08-27T16:22:00.692 INFO:teuthology.lock.ops:Checking smithi123.front.sepia.ceph.com
2023-08-27T16:22:00.693 INFO:teuthology.lock.ops:New key found. Updating...
2023-08-27T16:22:00.718 INFO:teuthology.lock.ops:Checking smithi149.front.sepia.ceph.com
2023-08-27T16:22:00.718 INFO:teuthology.lock.ops:New key found. Updating...
2023-08-27T16:22:00.735 INFO:teuthology.lock.ops:Checking smithi167.front.sepia.ceph.com
2023-08-27T16:22:00.736 INFO:teuthology.lock.ops:New key found. Updating...
2023-08-27T16:22:00.761 INFO:teuthology.lock.ops:Updating [smithi123.front.sepia.ceph.com]: set os type and version on server
2023-08-27T16:22:00.761 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com'
2023-08-27T16:22:00.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:22:00.828 DEBUG:teuthology.orchestra.run.smithi123:> true
2023-08-27T16:22:00.894 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com'
2023-08-27T16:22:00.894 DEBUG:teuthology.orchestra.run.smithi123:> uname -m
2023-08-27T16:22:00.949 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64
2023-08-27T16:22:00.949 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release
2023-08-27T16:22:01.004 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="Red Hat Enterprise Linux"
2023-08-27T16:22:01.004 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8.6 (Ootpa)"
2023-08-27T16:22:01.004 INFO:teuthology.orchestra.run.smithi123.stdout:ID="rhel"
2023-08-27T16:22:01.004 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="fedora"
2023-08-27T16:22:01.004 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8.6"
2023-08-27T16:22:01.005 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8"
2023-08-27T16:22:01.005 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-08-27T16:22:01.005 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31"
2023-08-27T16:22:01.005 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-08-27T16:22:01.005 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://www.redhat.com/"
2023-08-27T16:22:01.005 INFO:teuthology.orchestra.run.smithi123.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-08-27T16:22:01.005 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-08-27T16:22:01.005 INFO:teuthology.orchestra.run.smithi123.stdout:
2023-08-27T16:22:01.005 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-08-27T16:22:01.005 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-08-27T16:22:01.005 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-08-27T16:22:01.005 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-08-27T16:22:01.006 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server
2023-08-27T16:22:01.026 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: set os type and version on server
2023-08-27T16:22:01.027 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com'
2023-08-27T16:22:01.027 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T16:22:01.142 ERROR:teuthology.orchestra.connection:Error authenticating with smithi149.front.sepia.ceph.com: Authentication failed.
2023-08-27T16:22:01.142 DEBUG:teuthology.orchestra.run.smithi149:> true
2023-08-27T16:22:01.143 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-08-27T17:22:10.146 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com'
2023-08-27T17:22:10.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T17:22:10.224 DEBUG:teuthology.orchestra.run.smithi149:> true
2023-08-27T17:22:10.294 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com'
2023-08-27T17:22:10.294 DEBUG:teuthology.orchestra.run.smithi149:> uname -m
2023-08-27T17:22:10.353 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64
2023-08-27T17:22:10.354 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release
2023-08-27T17:22:10.412 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="Red Hat Enterprise Linux"
2023-08-27T17:22:10.412 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="8.6 (Ootpa)"
2023-08-27T17:22:10.412 INFO:teuthology.orchestra.run.smithi149.stdout:ID="rhel"
2023-08-27T17:22:10.412 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="fedora"
2023-08-27T17:22:10.412 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="8.6"
2023-08-27T17:22:10.412 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el8"
2023-08-27T17:22:10.412 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-08-27T17:22:10.412 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31"
2023-08-27T17:22:10.412 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-08-27T17:22:10.413 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://www.redhat.com/"
2023-08-27T17:22:10.413 INFO:teuthology.orchestra.run.smithi149.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-08-27T17:22:10.413 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-08-27T17:22:10.413 INFO:teuthology.orchestra.run.smithi149.stdout:
2023-08-27T17:22:10.413 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-08-27T17:22:10.413 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-08-27T17:22:10.413 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-08-27T17:22:10.413 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-08-27T17:22:10.414 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server
2023-08-27T17:22:10.435 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: set os type and version on server
2023-08-27T17:22:10.435 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com'
2023-08-27T17:22:10.435 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T17:22:13.519 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-08-27T17:22:22.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com'
2023-08-27T17:22:22.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T17:22:25.579 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-08-27T17:22:37.580 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com'
2023-08-27T17:22:37.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T17:22:40.651 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-08-27T17:22:43.652 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com'
2023-08-27T17:22:43.652 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-08-27T17:22:43.727 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167
2023-08-27T17:22:43.727 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds
2023-08-27T17:22:43.727 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/lock/ops.py", line 327, in reimage_machines
    update_nodes(reimaged)
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/lock/ops.py", line 38, in update_nodes
    inventory_info = remote.inventory_info
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 676, in inventory_info
    node['arch'] = self.arch
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 335, in arch
    self._arch = self.sh('uname -m').strip()
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 96, in sh
    proc = self.run(**kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 521, in run
    raise ConnectionError(f'Failed to reconnect to {self.shortname}')
ConnectionError: Failed to reconnect to smithi167
2023-08-27T17:22:43.728 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-08-27T17:22:43.864 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=dcb942defca346feb1d3eb35eb3dc92f
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/lock/ops.py", line 327, in reimage_machines
    update_nodes(reimaged)
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/lock/ops.py", line 38, in update_nodes
    inventory_info = remote.inventory_info
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 676, in inventory_info
    node['arch'] = self.arch
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 335, in arch
    self._arch = self.sh('uname -m').strip()
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 96, in sh
    proc = self.run(**kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 521, in run
    raise ConnectionError(f'Failed to reconnect to {self.shortname}')
ConnectionError: Failed to reconnect to smithi167
2023-08-27T17:22:43.865 INFO:teuthology.nuke:Checking targets against current locks
2023-08-27T17:22:43.930 DEBUG:teuthology.nuke:shortname: smithi123
2023-08-27T17:22:43.931 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-08-27T17:22:43.937 INFO:teuthology.nuke:Not nuking smithi149.front.sepia.ceph.com because description doesn't match: rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi != /home/teuthworker/archive/teuthology-2023-08-26_14:23:13-upgrade:pacific-x-reef-distro-default-smithi/7381715
2023-08-27T17:22:43.947 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi/7381667', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-27 16:16:24.890988', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}
2023-08-27T17:22:43.986 DEBUG:teuthology.nuke:shortname: smithi167
2023-08-27T17:22:43.986 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-08-27T17:22:43.995 INFO:teuthology.orchestra.console.smithi123:Power off
2023-08-27T17:22:43.995 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-27T17:22:44.016 DEBUG:teuthology.orchestra.console.smithi123:power off output: Chassis Power Control: Down/Off
2023-08-27T17:22:44.016 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-27T17:22:44.024 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-26_02:16:12-fs-wip-rishabh-xiubo-wip-57154-testing-default-smithi/7381664', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-27 16:14:33.396594', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}
2023-08-27T17:22:44.035 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on
2023-08-27T17:22:44.075 INFO:teuthology.orchestra.console.smithi167:Power off
2023-08-27T17:22:44.075 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-08-27T17:22:44.094 DEBUG:teuthology.orchestra.console.smithi167:power off output: Chassis Power Control: Down/Off
2023-08-27T17:22:44.095 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-27T17:22:44.115 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off
2023-08-27T17:22:44.216 INFO:teuthology.orchestra.console.smithi167:Power off completed
2023-08-27T17:22:44.365 INFO:teuthology.lock.ops:unlocked: smithi167.front.sepia.ceph.com
2023-08-27T17:22:48.036 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-27T17:22:48.146 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on
2023-08-27T17:22:52.147 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-27T17:22:52.256 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on
2023-08-27T17:22:56.257 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-08-27T17:22:56.366 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off
2023-08-27T17:22:56.467 INFO:teuthology.orchestra.console.smithi123:Power off completed
2023-08-27T17:22:56.621 INFO:teuthology.lock.ops:unlocked: smithi123.front.sepia.ceph.com
2023-08-27T17:22:56.622 ERROR:teuthology.nuke:Could not nuke the following targets:
targets:
  smithi149.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+r1jMQm994Gjc9rydhaWbZb4wBHmU8CR1z59/ZtDHAIB2Cabm62WCmB/0ZpVfd7qv0LprQixCUSEHkpCinANgPNDFqFT4CVmRFSBiwcIbtkRwA4Rbh+5UniuA5bhmVlP7jkaRZGUf8nwU0T+mg7zNg/HXSXt+UQErtVPoEEL2QYNjA3v1hf7Oe0xdykx33D5NcWrdkH2yyc/nQxl5kDtHM2p8/AG7DOSPfI7nMfXBgs7Fbbgk5fnRHMqwANZjEGYiPcTVxBs8tUNQmvL5cagWKjDGFjbrtfNHqhBRHMWkg41M6Hvpd3csc6AG00PyiKEWhbmmJZd2v7MHQvFJ9kW5E03jXm+Ygf6YkZJIHrJR2kc0UmpDo5/MqPwjH6qqNeI2c24aJtVhcWSP6jWtd9hqqwrAoyRopSYJKdMX9PMsU+H9lxE5urSuA3qdJZJjtiI53gH4OZ4v1H+j1ro80iOp0AjVQRp8WD+rK7NAcgwcWiKTfE2MtWvFilD1tEU2HKU=
2023-08-27T17:22:57.064 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/lock/ops.py", line 327, in reimage_machines
    update_nodes(reimaged)
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/lock/ops.py", line 38, in update_nodes
    inventory_info = remote.inventory_info
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 676, in inventory_info
    node['arch'] = self.arch
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 335, in arch
    self._arch = self.sh('uname -m').strip()
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 96, in sh
    proc = self.run(**kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 521, in run
    raise ConnectionError(f'Failed to reconnect to {self.shortname}')
ConnectionError: Failed to reconnect to smithi167