2023-07-17T19:41:33.420 INFO:root:teuthology version: 0.0.1.dev147+gff9991b 2023-07-17T19:41:33.422 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-17T19:41:33.422 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-17T19:41:33.632 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9172bd68b0>, signals=[15]) 2023-07-17T19:41:33.633 INFO:teuthology.lock.ops:Start node 'smithi087.front.sepia.ceph.com' reimaging 2023-07-17T19:41:33.634 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: reset os type and version on server 2023-07-17T19:41:33.634 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2023-07-17T19:41:33.656 INFO:teuthology.lock.ops:Start node 'smithi181.front.sepia.ceph.com' reimaging 2023-07-17T19:41:33.656 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: reset os type and version on server 2023-07-17T19:41:33.657 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-07-17T19:41:33.658 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /host {"name": "smithi087"} 2023-07-17T19:41:43.694 INFO:teuthology.lock.ops:Start node 'smithi182.front.sepia.ceph.com' reimaging 2023-07-17T19:41:43.694 INFO:teuthology.lock.ops:Updating [smithi182.front.sepia.ceph.com]: reset os type and version on server 2023-07-17T19:41:43.694 INFO:teuthology.lock.ops:Updating smithi182.front.sepia.ceph.com on lock server 2023-07-17T19:41:43.696 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /host {"name": "smithi181"} 2023-07-17T19:41:56.362 DEBUG:teuthology.provision.fog.smithi182:FOG request: GET /host {"name": "smithi182"} 2023-07-17T19:42:02.064 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-07-17T19:42:10.499 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-07-17T19:42:20.509 DEBUG:teuthology.provision.fog.smithi087:Requesting image smithi_rhel_8.4 (ID 7) 2023-07-17T19:42:20.510 DEBUG:teuthology.provision.fog.smithi087:FOG request: PUT /host/192 {"imageID": 7} 2023-07-17T19:42:25.517 DEBUG:teuthology.provision.fog.smithi182:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-07-17T19:42:30.522 DEBUG:teuthology.provision.fog.smithi181:Requesting image smithi_rhel_8.4 (ID 7) 2023-07-17T19:42:30.522 DEBUG:teuthology.provision.fog.smithi181:FOG request: PUT /host/286 {"imageID": 7} 2023-07-17T19:42:31.409 INFO:teuthology.provision.fog.smithi087:Scheduling deploy of rhel 8.4 2023-07-17T19:42:31.409 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-07-17T19:42:31.465 DEBUG:teuthology.provision.fog.smithi182:Requesting image smithi_rhel_8.4 (ID 7) 2023-07-17T19:42:31.466 DEBUG:teuthology.provision.fog.smithi182:FOG request: PUT /host/287 {"imageID": 7} 2023-07-17T19:42:31.473 INFO:teuthology.provision.fog.smithi181:Scheduling deploy of rhel 8.4 2023-07-17T19:42:31.474 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-07-17T19:42:31.501 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /tasktype {"name": "deploy"} 2023-07-17T19:42:36.512 DEBUG:teuthology.provision.fog.smithi181:Canceling deploy task with ID 392495 2023-07-17T19:42:36.513 DEBUG:teuthology.provision.fog.smithi181:FOG request: DELETE /task/cancel {"id": 392495} 2023-07-17T19:42:36.514 INFO:teuthology.provision.fog.smithi182:Scheduling deploy of rhel 8.4 2023-07-17T19:42:36.514 DEBUG:teuthology.provision.fog.smithi182:FOG request: GET /task/active None 2023-07-17T19:42:46.568 DEBUG:teuthology.provision.fog.smithi087:FOG request: POST /host/192/task {"taskTypeID": 1} 2023-07-17T19:42:51.577 DEBUG:teuthology.provision.fog.smithi182:Canceling deploy task with ID 392496 2023-07-17T19:42:51.577 DEBUG:teuthology.provision.fog.smithi182:FOG request: DELETE /task/cancel {"id": 392496} 2023-07-17T19:42:51.579 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /tasktype {"name": "deploy"} 2023-07-17T19:43:01.625 DEBUG:teuthology.provision.fog.smithi181:FOG request: POST /host/286/task {"taskTypeID": 1} 2023-07-17T19:43:04.960 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-07-17T19:43:04.961 DEBUG:teuthology.provision.fog.smithi182:FOG request: GET /tasktype {"name": "deploy"} 2023-07-17T19:43:05.013 DEBUG:teuthology.provision.fog.smithi182:FOG request: POST /host/287/task {"taskTypeID": 1} 2023-07-17T19:43:05.018 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-07-17T19:43:10.059 INFO:teuthology.orchestra.console:Power off smithi087 2023-07-17T19:43:10.060 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-17T19:43:10.083 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:11.744 DEBUG:teuthology.provision.fog.smithi182:FOG request: GET /task/active None 2023-07-17T19:43:16.780 INFO:teuthology.orchestra.console:Power off smithi181 2023-07-17T19:43:16.781 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-17T19:43:21.809 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:21.852 INFO:teuthology.orchestra.console:Power off smithi182 2023-07-17T19:43:21.853 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-17T19:43:21.935 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:22.020 INFO:teuthology.orchestra.console:Power off for smithi087 completed 2023-07-17T19:43:22.121 INFO:teuthology.orchestra.console:Power on smithi087 2023-07-17T19:43:22.121 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-17T19:43:26.948 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:31.887 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:32.059 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:32.152 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:36.174 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:36.898 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:37.172 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:41.284 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:42.008 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:42.285 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:46.395 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:47.120 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:47.396 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:51.508 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:52.233 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:52.511 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:52.721 INFO:teuthology.orchestra.console:Power off for smithi181 completed 2023-07-17T19:43:52.822 INFO:teuthology.orchestra.console:Power on smithi181 2023-07-17T19:43:52.823 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-17T19:43:56.620 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:43:57.346 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:44:01.732 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:44:01.945 INFO:teuthology.orchestra.console:Power on for smithi087 completed 2023-07-17T19:44:02.046 INFO:teuthology.provision.fog.smithi087:Waiting for deploy to finish 2023-07-17T19:44:02.047 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-07-17T19:44:05.466 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:44:07.855 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:44:10.577 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:44:11.880 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:44:15.691 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:44:16.993 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:44:20.521 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-07-17T19:44:25.529 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:44:25.537 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:44:30.641 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:44:30.649 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:44:35.752 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:44:35.760 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:44:35.971 INFO:teuthology.orchestra.console:Power on for smithi181 completed 2023-07-17T19:44:36.072 INFO:teuthology.provision.fog.smithi181:Waiting for deploy to finish 2023-07-17T19:44:36.072 DEBUG:teuthology.provision.fog.smithi181:FOG request: GET /task/active None 2023-07-17T19:44:39.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:44:44.247 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-07-17T19:44:44.249 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:44:47.316 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-07-17T19:44:49.361 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:44:53.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:44:58.326 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:44:59.316 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-07-17T19:44:59.841 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds 2023-07-17T19:44:59.842 ERROR:teuthology.orchestra.console:Failed to power off smithi182 2023-07-17T19:44:59.842 INFO:teuthology.orchestra.console:Power on smithi182 2023-07-17T19:44:59.842 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-17T19:45:01.396 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-07-17T19:45:07.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:45:08.510 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:45:08.634 INFO:teuthology.orchestra.console:Power on for smithi182 completed 2023-07-17T19:45:08.735 INFO:teuthology.provision.fog.smithi182:Waiting for deploy to finish 2023-07-17T19:45:08.736 DEBUG:teuthology.provision.fog.smithi182:FOG request: GET /task/active None 2023-07-17T19:45:08.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi182.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:45:09.460 WARNING:teuthology.provision.fog.smithi182:[Errno None] Unable to connect to port 22 on 172.21.15.182 2023-07-17T19:45:10.485 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-07-17T19:45:14.916 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-07-17T19:45:19.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi182.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:45:24.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:45:29.928 WARNING:teuthology.provision.fog.smithi182:[Errno None] Unable to connect to port 22 on 172.21.15.182 2023-07-17T19:45:35.932 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi182.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:45:44.016 WARNING:teuthology.provision.fog.smithi182:[Errno None] Unable to connect to port 22 on 172.21.15.182 2023-07-17T19:45:44.998 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-07-17T19:45:48.408 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-07-17T19:45:50.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi182.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:45:55.025 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:46:00.139 ERROR:teuthology.orchestra.connection:Error authenticating with smithi181.front.sepia.ceph.com: Authentication failed. 2023-07-17T19:46:00.141 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-17T19:46:00.142 DEBUG:teuthology.orchestra.run.smithi181:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-17T19:46:00.144 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:00.265 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-07-17T19:46:01.154 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:02.160 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:03.166 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:04.171 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:05.177 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:06.181 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:07.186 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:08.192 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:09.201 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:10.206 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:11.212 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:12.219 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:13.224 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:13.488 WARNING:teuthology.provision.fog.smithi182:[Errno None] Unable to connect to port 22 on 172.21.15.182 2023-07-17T19:46:14.230 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:15.235 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:15.328 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-07-17T19:46:15.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:46:16.241 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:17.247 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:18.252 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:18.480 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-07-17T19:46:19.258 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:19.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi182.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:46:19.566 ERROR:teuthology.orchestra.connection:Error authenticating with smithi182.front.sepia.ceph.com: Authentication failed. 2023-07-17T19:46:19.568 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-17T19:46:19.568 DEBUG:teuthology.orchestra.run.smithi182:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-17T19:46:19.570 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:20.266 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:20.579 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:21.275 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:21.588 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:22.284 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:22.597 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:23.294 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:23.607 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:24.308 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:24.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:46:29.490 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:29.490 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:30.496 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:30.496 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:31.503 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:31.503 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:32.510 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:32.510 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:32.560 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-07-17T19:46:33.517 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:33.518 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:34.523 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:34.524 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:35.529 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:35.530 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:36.535 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:36.536 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:37.541 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:37.541 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:38.547 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:38.547 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:38.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:46:43.568 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:43.568 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:44.574 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:44.575 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:45.583 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:45.583 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:46.592 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:46.593 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:46.644 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-07-17T19:46:47.599 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:47.600 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:48.607 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:48.608 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:49.614 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:49.614 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:50.621 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:50.621 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:51.627 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:51.628 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:52.634 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:52.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:46:57.650 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:58.662 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:46:59.668 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:47:00.679 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:47:00.720 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-07-17T19:47:01.689 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:47:02.697 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:47:03.705 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:47:04.712 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:47:05.720 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:47:06.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:47:11.725 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:47:12.731 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:47:13.738 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-17T19:48:11.725 WARNING:teuthology.provision.fog.smithi087:timed out 2023-07-17T19:48:17.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:48:23.116 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-17T19:48:23.117 DEBUG:teuthology.orchestra.run.smithi087:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-17T19:48:23.378 INFO:teuthology.provision.fog.smithi087:Node is ready 2023-07-17T19:48:23.379 DEBUG:teuthology.orchestra.run.smithi087:> hostname 2023-07-17T19:48:23.397 INFO:teuthology.orchestra.run.smithi087.stdout:smithi087.front.sepia.ceph.com 2023-07-17T19:48:23.398 DEBUG:teuthology.orchestra.run.smithi087:> grep smithi087.front.sepia.ceph.com /etc/hosts 2023-07-17T19:48:23.453 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87 smithi087.front.sepia.ceph.com smithi087 2023-07-17T19:48:23.454 DEBUG:teuthology.orchestra.run.smithi087:> sudo hostname smithi087 2023-07-17T19:48:23.561 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/smithi087.front.sepia.ceph.com/smithi087/g' /etc/hosts 2023-07-17T19:48:23.586 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/172.21.15.87/172.21.15.87/g' /etc/hosts 2023-07-17T19:48:23.651 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/smithi087.front.sepia.ceph.com/smithi087/g' /etc/hostname 2023-07-17T19:48:23.715 DEBUG:teuthology.orchestra.run.smithi087:> sudo hostname smithi087 2023-07-17T19:48:23.778 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2023-07-17T19:48:23.833 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="Red Hat Enterprise Linux" 2023-07-17T19:48:23.834 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="8.4 (Ootpa)" 2023-07-17T19:48:23.834 INFO:teuthology.orchestra.run.smithi087.stdout:ID="rhel" 2023-07-17T19:48:23.834 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="fedora" 2023-07-17T19:48:23.834 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="8.4" 2023-07-17T19:48:23.835 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el8" 2023-07-17T19:48:23.835 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-07-17T19:48:23.835 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2023-07-17T19:48:23.835 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-07-17T19:48:23.835 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://www.redhat.com/" 2023-07-17T19:48:23.835 INFO:teuthology.orchestra.run.smithi087.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-17T19:48:23.836 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-17T19:48:23.836 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-07-17T19:48:23.836 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-17T19:48:23.836 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-07-17T19:48:23.836 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-17T19:48:23.836 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-07-17T19:48:23.837 INFO:teuthology.provision.fog.smithi087:Deploy complete! 2023-07-17T19:48:23.838 DEBUG:teuthology.parallel:result is None 2023-07-17T19:48:40.688 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-07-17T19:48:40.690 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi181 2023-07-17T19:48:49.693 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-07-17T19:48:49.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:48:50.041 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-07-17T19:48:50.288 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-07-17T19:48:50.288 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-17T19:48:50.289 DEBUG:teuthology.orchestra.run.smithi181:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-17T19:49:00.345 INFO:teuthology.provision.fog.smithi181:Node is ready 2023-07-17T19:49:00.345 DEBUG:teuthology.orchestra.run.smithi181:> hostname 2023-07-17T19:49:00.360 INFO:teuthology.orchestra.run.smithi181.stdout:smithi181 2023-07-17T19:49:00.361 DEBUG:teuthology.orchestra.run.smithi181:> grep smithi181 /etc/hosts 2023-07-17T19:49:00.414 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181 smithi181.front.sepia.ceph.com smithi181 2023-07-17T19:49:00.415 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181 2023-07-17T19:49:00.564 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181/smithi181/g' /etc/hosts 2023-07-17T19:49:00.589 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/172.21.15.181/172.21.15.181/g' /etc/hosts 2023-07-17T19:49:00.656 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181/smithi181/g' /etc/hostname 2023-07-17T19:49:00.730 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181 2023-07-17T19:49:00.792 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-07-17T19:49:00.846 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="Red Hat Enterprise Linux" 2023-07-17T19:49:00.846 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8.4 (Ootpa)" 2023-07-17T19:49:00.846 INFO:teuthology.orchestra.run.smithi181.stdout:ID="rhel" 2023-07-17T19:49:00.847 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="fedora" 2023-07-17T19:49:00.847 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8.4" 2023-07-17T19:49:00.847 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2023-07-17T19:49:00.847 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-07-17T19:49:00.848 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-07-17T19:49:00.848 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-07-17T19:49:00.848 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.redhat.com/" 2023-07-17T19:49:00.848 INFO:teuthology.orchestra.run.smithi181.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-17T19:49:00.849 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-17T19:49:00.849 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-07-17T19:49:00.849 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-17T19:49:00.849 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-07-17T19:49:00.850 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-17T19:49:00.850 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-07-17T19:49:00.850 INFO:teuthology.provision.fog.smithi181:Deploy complete! 2023-07-17T19:49:00.852 DEBUG:teuthology.parallel:result is None 2023-07-17T19:49:01.168 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-07-17T19:49:01.169 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi182 2023-07-17T19:49:10.170 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi182.front.sepia.ceph.com' 2023-07-17T19:49:10.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi182.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:49:10.498 DEBUG:teuthology.orchestra.run.smithi182:> true 2023-07-17T19:49:10.806 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi182.front.sepia.ceph.com' 2023-07-17T19:49:10.806 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-17T19:49:10.806 DEBUG:teuthology.orchestra.run.smithi182:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-17T19:49:20.823 INFO:teuthology.provision.fog.smithi182:Node is ready 2023-07-17T19:49:20.823 DEBUG:teuthology.orchestra.run.smithi182:> hostname 2023-07-17T19:49:20.838 INFO:teuthology.orchestra.run.smithi182.stdout:smithi182 2023-07-17T19:49:20.838 DEBUG:teuthology.orchestra.run.smithi182:> grep smithi182 /etc/hosts 2023-07-17T19:49:20.891 INFO:teuthology.orchestra.run.smithi182.stdout:172.21.15.182 smithi182.front.sepia.ceph.com smithi182 2023-07-17T19:49:20.892 DEBUG:teuthology.orchestra.run.smithi182:> sudo hostname smithi182 2023-07-17T19:49:20.957 DEBUG:teuthology.orchestra.run.smithi182:> sudo sed -i -e 's/smithi182/smithi182/g' /etc/hosts 2023-07-17T19:49:21.021 DEBUG:teuthology.orchestra.run.smithi182:> sudo sed -i -e 's/172.21.15.182/172.21.15.182/g' /etc/hosts 2023-07-17T19:49:21.086 DEBUG:teuthology.orchestra.run.smithi182:> sudo sed -i -e 's/smithi182/smithi182/g' /etc/hostname 2023-07-17T19:49:21.150 DEBUG:teuthology.orchestra.run.smithi182:> sudo hostname smithi182 2023-07-17T19:49:21.217 DEBUG:teuthology.orchestra.run.smithi182:> cat /etc/os-release 2023-07-17T19:49:21.275 INFO:teuthology.orchestra.run.smithi182.stdout:NAME="Red Hat Enterprise Linux" 2023-07-17T19:49:21.275 INFO:teuthology.orchestra.run.smithi182.stdout:VERSION="8.4 (Ootpa)" 2023-07-17T19:49:21.275 INFO:teuthology.orchestra.run.smithi182.stdout:ID="rhel" 2023-07-17T19:49:21.276 INFO:teuthology.orchestra.run.smithi182.stdout:ID_LIKE="fedora" 2023-07-17T19:49:21.276 INFO:teuthology.orchestra.run.smithi182.stdout:VERSION_ID="8.4" 2023-07-17T19:49:21.276 INFO:teuthology.orchestra.run.smithi182.stdout:PLATFORM_ID="platform:el8" 2023-07-17T19:49:21.276 INFO:teuthology.orchestra.run.smithi182.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-07-17T19:49:21.276 INFO:teuthology.orchestra.run.smithi182.stdout:ANSI_COLOR="0;31" 2023-07-17T19:49:21.276 INFO:teuthology.orchestra.run.smithi182.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-07-17T19:49:21.276 INFO:teuthology.orchestra.run.smithi182.stdout:HOME_URL="https://www.redhat.com/" 2023-07-17T19:49:21.277 INFO:teuthology.orchestra.run.smithi182.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-17T19:49:21.277 INFO:teuthology.orchestra.run.smithi182.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-17T19:49:21.277 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-17T19:49:21.278 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-17T19:49:21.278 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-07-17T19:49:21.278 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-17T19:49:21.278 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-07-17T19:49:21.278 INFO:teuthology.provision.fog.smithi182:Deploy complete! 2023-07-17T19:49:21.279 DEBUG:teuthology.parallel:result is None 2023-07-17T19:49:41.523 INFO:teuthology.lock.ops:Checking smithi087.front.sepia.ceph.com 2023-07-17T19:49:41.524 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-17T19:49:46.554 INFO:teuthology.lock.ops:Checking smithi181.front.sepia.ceph.com 2023-07-17T19:49:46.554 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-17T19:49:51.586 INFO:teuthology.lock.ops:Checking smithi182.front.sepia.ceph.com 2023-07-17T19:49:51.586 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-17T19:49:56.612 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: set os type and version on server 2023-07-17T19:49:56.612 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-07-17T19:49:56.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:50:01.684 DEBUG:teuthology.orchestra.run.smithi087:> true 2023-07-17T19:50:01.802 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-07-17T19:50:01.803 DEBUG:teuthology.orchestra.run.smithi087:> uname -m 2023-07-17T19:50:01.857 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2023-07-17T19:50:01.858 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2023-07-17T19:50:01.912 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="Red Hat Enterprise Linux" 2023-07-17T19:50:01.912 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="8.4 (Ootpa)" 2023-07-17T19:50:01.912 INFO:teuthology.orchestra.run.smithi087.stdout:ID="rhel" 2023-07-17T19:50:01.913 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="fedora" 2023-07-17T19:50:01.913 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="8.4" 2023-07-17T19:50:01.913 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el8" 2023-07-17T19:50:01.913 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-07-17T19:50:01.913 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2023-07-17T19:50:01.913 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-07-17T19:50:01.914 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://www.redhat.com/" 2023-07-17T19:50:01.914 INFO:teuthology.orchestra.run.smithi087.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-17T19:50:01.914 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-17T19:50:01.916 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-07-17T19:50:01.916 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-17T19:50:01.916 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-07-17T19:50:01.916 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-17T19:50:01.916 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-07-17T19:50:01.917 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2023-07-17T19:50:06.950 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: set os type and version on server 2023-07-17T19:50:06.950 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-07-17T19:50:06.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:50:07.163 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-07-17T19:50:07.250 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-07-17T19:50:07.250 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2023-07-17T19:50:07.304 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2023-07-17T19:50:07.304 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-07-17T19:50:07.358 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="Red Hat Enterprise Linux" 2023-07-17T19:50:07.358 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8.4 (Ootpa)" 2023-07-17T19:50:07.358 INFO:teuthology.orchestra.run.smithi181.stdout:ID="rhel" 2023-07-17T19:50:07.358 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="fedora" 2023-07-17T19:50:07.358 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8.4" 2023-07-17T19:50:07.358 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2023-07-17T19:50:07.359 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-07-17T19:50:07.359 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-07-17T19:50:07.359 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-07-17T19:50:07.359 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.redhat.com/" 2023-07-17T19:50:07.359 INFO:teuthology.orchestra.run.smithi181.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-17T19:50:07.359 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-17T19:50:07.359 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-07-17T19:50:07.359 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-17T19:50:07.360 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-07-17T19:50:07.360 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-17T19:50:07.360 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-07-17T19:50:07.360 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-07-17T19:50:12.399 INFO:teuthology.lock.ops:Updating [smithi182.front.sepia.ceph.com]: set os type and version on server 2023-07-17T19:50:12.399 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi182.front.sepia.ceph.com' 2023-07-17T19:50:12.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi182.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-17T19:50:17.470 DEBUG:teuthology.orchestra.run.smithi182:> true 2023-07-17T19:50:17.538 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi182.front.sepia.ceph.com' 2023-07-17T19:50:17.539 DEBUG:teuthology.orchestra.run.smithi182:> uname -m 2023-07-17T19:50:17.593 INFO:teuthology.orchestra.run.smithi182.stdout:x86_64 2023-07-17T19:50:17.594 DEBUG:teuthology.orchestra.run.smithi182:> cat /etc/os-release 2023-07-17T19:50:17.647 INFO:teuthology.orchestra.run.smithi182.stdout:NAME="Red Hat Enterprise Linux" 2023-07-17T19:50:17.647 INFO:teuthology.orchestra.run.smithi182.stdout:VERSION="8.4 (Ootpa)" 2023-07-17T19:50:17.647 INFO:teuthology.orchestra.run.smithi182.stdout:ID="rhel" 2023-07-17T19:50:17.648 INFO:teuthology.orchestra.run.smithi182.stdout:ID_LIKE="fedora" 2023-07-17T19:50:17.648 INFO:teuthology.orchestra.run.smithi182.stdout:VERSION_ID="8.4" 2023-07-17T19:50:17.648 INFO:teuthology.orchestra.run.smithi182.stdout:PLATFORM_ID="platform:el8" 2023-07-17T19:50:17.648 INFO:teuthology.orchestra.run.smithi182.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-07-17T19:50:17.648 INFO:teuthology.orchestra.run.smithi182.stdout:ANSI_COLOR="0;31" 2023-07-17T19:50:17.648 INFO:teuthology.orchestra.run.smithi182.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-07-17T19:50:17.649 INFO:teuthology.orchestra.run.smithi182.stdout:HOME_URL="https://www.redhat.com/" 2023-07-17T19:50:17.649 INFO:teuthology.orchestra.run.smithi182.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-17T19:50:17.649 INFO:teuthology.orchestra.run.smithi182.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-17T19:50:17.649 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-07-17T19:50:17.649 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-17T19:50:17.649 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-07-17T19:50:17.649 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-17T19:50:17.650 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-07-17T19:50:17.650 INFO:teuthology.lock.ops:Updating smithi182.front.sepia.ceph.com on lock server 2023-07-17T19:50:22.682 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-17T19:50:22.682 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-17T19:50:27.789 INFO:teuthology.dispatcher.supervisor:Running job 7341826 2023-07-17T19:50:27.789 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-07-17_15:50:05-fs-wip-yuri3-testing-2023-07-14-0724-quincy-distro-default-smithi/7341826 --name yuriw-2023-07-17_15:50:05-fs-wip-yuri3-testing-2023-07-14-0724-quincy-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{yes}} -- /home/teuthworker/archive/yuriw-2023-07-17_15:50:05-fs-wip-yuri3-testing-2023-07-14-0724-quincy-distro-default-smithi/7341826/orig.config.yaml 2023-07-17T19:50:27.793 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-07-17_15:50:05-fs-wip-yuri3-testing-2023-07-14-0724-quincy-distro-default-smithi/7341826 2023-07-17T19:50:27.794 INFO:teuthology.dispatcher.supervisor:Job PID: 31034 2023-07-17T19:50:27.794 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-17T19:52:27.894 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-17T19:52:27.895 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-17T19:54:33.073 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-17T19:54:33.074 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-17T19:54:38.154 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-07-17T19:54:50.069 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi181 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-07-17_15:50:05-fs-wip-yuri3-testing-2023-07-14-0724-quincy-distro-default-smithi/7341827 2023-07-17T19:54:50.070 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi182 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-07-17_15:50:05-fs-wip-yuri3-testing-2023-07-14-0724-quincy-distro-default-smithi/7341827 2023-07-17T19:54:50.070 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-07-17T19:54:50.070 INFO:teuthology.nuke:Checking targets against current locks 2023-07-17T19:55:05.134 DEBUG:teuthology.nuke:shortname: smithi087 2023-07-17T19:55:05.135 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-17T19:55:25.187 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-17_15:50:05-fs-wip-yuri3-testing-2023-07-14-0724-quincy-distro-default-smithi/7341826', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-17 19:41:32.575095', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKcScQuAGPzZ5x8Q3PB6UEx0SRM6dZT/RGzT/v9tb9USPlpPfUyoph5/7bt/lNyyhJfollnnE7ahsBahYSBdOag='} 2023-07-17T19:55:25.188 DEBUG:teuthology.nuke:shortname: smithi181 2023-07-17T19:55:25.189 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-17T19:55:35.224 DEBUG:teuthology.nuke:shortname: smithi182 2023-07-17T19:55:35.225 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-17T19:55:39.750 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-17_15:50:05-fs-wip-yuri3-testing-2023-07-14-0724-quincy-distro-default-smithi/7341827', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-17 19:41:33.032557', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCshwaR56BM2a2mQO5JJdDPyK4Foj97Um5RgqPFkS93dcTJ6UKC6M9joTWEgUWgWTDljA2KqtN8kR8ea+TrQM/kQEM7vVPdJ+ePCiPRFs3FNt5/SnWBT0dQWxeZfaDb95FbPHoSGnj2JULooOSfhAyaHnB9FbgtI1WEriW5MosH2YJCH6v9AYj/hFBvwSaUI/t4UQnAaixEpCBM6Q+s9orKuRQ0OAWAECvdgPgRpFGdgyk2Y999u7MolR+b22helKTMICz2J2LTUF4uCN76LZQAP77rWqQXacieqEirgS+b8OL+Mz9UvQOd2sMOZvpWpnjgmaM/ixXE2KbViVuCr4OkmLuPRahnfvb7+L4zh55yaCS7nJyOdrjriSCccgtsdXhtMM+yrv71zziBKO6IgJ2UrEqX/VKe++NtM5292SHnW6so8/M23ySiq0Z3MP0TlLroTlp4MiGj63rk6E5nTgNN0ZI+iTQGHiaOhERBin73UAVHIwL7QZ9VvwaR7huoquk='} 2023-07-17T19:55:39.779 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi182.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-17_15:50:05-fs-wip-yuri3-testing-2023-07-14-0724-quincy-distro-default-smithi/7341827', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-17 19:41:33.034304', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMuIFlO+qXAXyD27e5MD2B4EbyUkOGDRkS8m5T1kFDDyBPPRRjI1QIr4WNJ//4qZMaLXYNj4ihl5BQ4GqhOsjl4='} 2023-07-17T19:55:39.789 INFO:teuthology.orchestra.console:Power off smithi087 2023-07-17T19:55:39.790 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-17T19:55:39.813 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:55:39.833 INFO:teuthology.orchestra.console:Power off smithi181 2023-07-17T19:55:39.834 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-17T19:55:39.857 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:55:39.877 INFO:teuthology.orchestra.console:Power off smithi182 2023-07-17T19:55:39.878 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-17T19:55:39.901 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi182.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-17T19:55:39.950 INFO:teuthology.orchestra.console:Power off for smithi087 completed 2023-07-17T19:55:39.989 INFO:teuthology.orchestra.console:Power off for smithi181 completed 2023-07-17T19:55:40.029 INFO:teuthology.orchestra.console:Power off for smithi182 completed 2023-07-17T19:55:40.111 INFO:teuthology.lock.ops:unlocked: smithi087.front.sepia.ceph.com 2023-07-17T19:55:40.136 INFO:teuthology.lock.ops:unlocked: smithi181.front.sepia.ceph.com 2023-07-17T19:55:40.178 INFO:teuthology.lock.ops:unlocked: smithi182.front.sepia.ceph.com