2023-07-28T09:46:26.841 INFO:root:teuthology version: 0.0.1.dev165+g3117e89
2023-07-28T09:46:26.843 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T09:46:26.843 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T09:46:47.011 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f539a59f850>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f5399c9b430>, signals=[15])
2023-07-28T09:46:47.012 INFO:teuthology.lock.ops:Start node 'smithi003.front.sepia.ceph.com' reimaging
2023-07-28T09:46:47.013 INFO:teuthology.lock.ops:Updating [smithi003.front.sepia.ceph.com]: reset os type and version on server
2023-07-28T09:46:47.013 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server
2023-07-28T09:46:57.198 INFO:teuthology.lock.ops:Start node 'smithi033.front.sepia.ceph.com' reimaging
2023-07-28T09:46:57.199 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: reset os type and version on server
2023-07-28T09:46:57.199 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server
2023-07-28T09:46:57.200 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /host {"name": "smithi003"}
2023-07-28T09:47:07.237 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /host {"name": "smithi033"}
2023-07-28T09:47:12.548 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-07-28T09:47:22.583 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-07-28T09:47:27.589 DEBUG:teuthology.provision.fog.smithi003:Requesting image smithi_rhel_8.6 (ID 5)
2023-07-28T09:47:27.589 DEBUG:teuthology.provision.fog.smithi003:FOG request: PUT /host/109 {"imageID": 5}
2023-07-28T09:47:32.636 DEBUG:teuthology.provision.fog.smithi033:Requesting image smithi_rhel_8.6 (ID 5)
2023-07-28T09:47:32.636 DEBUG:teuthology.provision.fog.smithi033:FOG request: PUT /host/138 {"imageID": 5}
2023-07-28T09:47:37.644 INFO:teuthology.provision.fog.smithi003:Scheduling deploy of rhel 8.6
2023-07-28T09:47:37.644 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None
2023-07-28T09:47:39.095 INFO:teuthology.provision.fog.smithi033:Scheduling deploy of rhel 8.6
2023-07-28T09:47:39.095 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None
2023-07-28T09:47:39.098 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /tasktype {"name": "deploy"}
2023-07-28T09:47:39.128 DEBUG:teuthology.provision.fog.smithi003:FOG request: POST /host/109/task {"taskTypeID": 1}
2023-07-28T09:47:39.136 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /tasktype {"name": "deploy"}
2023-07-28T09:47:39.166 DEBUG:teuthology.provision.fog.smithi033:FOG request: POST /host/138/task {"taskTypeID": 1}
2023-07-28T09:47:39.176 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None
2023-07-28T09:47:44.231 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None
2023-07-28T09:47:44.267 INFO:teuthology.orchestra.console.smithi003:Power off
2023-07-28T09:47:44.268 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-28T09:47:49.298 DEBUG:teuthology.orchestra.console.smithi003:power off output: Chassis Power Control: Down/Off
2023-07-28T09:47:49.299 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:47:49.330 INFO:teuthology.orchestra.console.smithi033:Power off
2023-07-28T09:47:49.330 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-28T09:47:50.309 DEBUG:teuthology.orchestra.console.smithi003:check power output: 
2023-07-28T09:47:54.314 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:47:54.353 DEBUG:teuthology.orchestra.console.smithi033:power off output: Chassis Power Control: Down/Off
2023-07-28T09:47:54.354 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:47:55.362 DEBUG:teuthology.orchestra.console.smithi033:check power output: 
2023-07-28T09:47:55.423 DEBUG:teuthology.orchestra.console.smithi003:check power output: 
2023-07-28T09:47:59.363 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:47:59.424 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:48:00.474 DEBUG:teuthology.orchestra.console.smithi033:check power output: 
2023-07-28T09:48:00.534 DEBUG:teuthology.orchestra.console.smithi003:check power output: 
2023-07-28T09:48:04.475 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:48:04.535 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:48:04.647 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off
2023-07-28T09:48:04.748 INFO:teuthology.orchestra.console.smithi003:Power off completed
2023-07-28T09:48:04.849 INFO:teuthology.orchestra.console.smithi003:Power on
2023-07-28T09:48:04.850 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-07-28T09:48:04.870 DEBUG:teuthology.orchestra.console.smithi003:power on output: Chassis Power Control: Up/On
2023-07-28T09:48:04.871 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:48:04.892 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off
2023-07-28T09:48:05.584 DEBUG:teuthology.orchestra.console.smithi033:check power output: 
2023-07-28T09:48:08.897 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:48:09.006 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off
2023-07-28T09:48:09.585 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:48:10.694 DEBUG:teuthology.orchestra.console.smithi033:check power output: 
2023-07-28T09:48:13.010 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:48:14.119 DEBUG:teuthology.orchestra.console.smithi003:check power output: 
2023-07-28T09:48:14.696 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:48:14.806 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off
2023-07-28T09:48:14.907 INFO:teuthology.orchestra.console.smithi033:Power off completed
2023-07-28T09:48:15.008 INFO:teuthology.orchestra.console.smithi033:Power on
2023-07-28T09:48:15.009 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-07-28T09:48:15.029 DEBUG:teuthology.orchestra.console.smithi033:power on output: Chassis Power Control: Up/On
2023-07-28T09:48:15.030 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:48:15.050 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off
2023-07-28T09:48:18.121 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:48:19.052 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:48:19.231 DEBUG:teuthology.orchestra.console.smithi003:check power output: 
2023-07-28T09:48:20.162 DEBUG:teuthology.orchestra.console.smithi033:check power output: 
2023-07-28T09:48:23.233 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:48:23.342 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on
2023-07-28T09:48:23.443 INFO:teuthology.orchestra.console.smithi003:Power on completed
2023-07-28T09:48:23.544 INFO:teuthology.provision.fog.smithi003:Waiting for deploy to finish
2023-07-28T09:48:23.544 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None
2023-07-28T09:48:24.163 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:48:25.272 DEBUG:teuthology.orchestra.console.smithi033:check power output: 
2023-07-28T09:48:29.277 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:48:30.386 DEBUG:teuthology.orchestra.console.smithi033:check power output: 
2023-07-28T09:48:34.387 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:48:35.496 DEBUG:teuthology.orchestra.console.smithi033:check power output: 
2023-07-28T09:48:38.614 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None
2023-07-28T09:48:39.498 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:48:40.607 DEBUG:teuthology.orchestra.console.smithi033:check power output: 
2023-07-28T09:48:44.609 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T09:48:44.719 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on
2023-07-28T09:48:44.820 INFO:teuthology.orchestra.console.smithi033:Power on completed
2023-07-28T09:48:44.921 INFO:teuthology.provision.fog.smithi033:Waiting for deploy to finish
2023-07-28T09:48:44.922 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None
2023-07-28T09:48:44.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-28T09:48:48.052 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33
2023-07-28T09:48:53.682 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None
2023-07-28T09:48:58.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-28T09:49:06.768 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33
2023-07-28T09:49:12.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-28T09:49:18.765 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None
2023-07-28T09:49:31.120 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33
2023-07-28T09:49:33.824 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None
2023-07-28T09:49:37.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-28T09:49:42.942 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33
2023-07-28T09:49:48.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-28T09:49:53.955 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None
2023-07-28T09:49:59.030 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: Authentication failed.
2023-07-28T09:49:59.032 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-28T09:49:59.032 DEBUG:teuthology.orchestra.run.smithi033:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-28T09:49:59.033 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:00.038 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:01.042 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:02.045 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:03.049 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:04.053 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:05.058 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:06.062 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:07.067 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:08.071 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:09.077 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:10.081 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:11.086 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:12.091 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:13.096 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:14.014 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None
2023-07-28T09:50:15.498 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:16.502 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:17.507 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:18.512 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:19.517 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:20.522 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:21.527 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:22.533 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:23.539 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:24.544 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:25.549 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:26.554 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:27.559 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:28.565 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:29.570 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:30.558 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None
2023-07-28T09:50:30.575 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:31.580 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:32.585 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:33.591 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:34.596 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:35.601 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:36.607 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:37.612 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:38.618 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:39.623 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:40.628 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:41.633 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:42.637 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:43.641 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:44.646 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-28T09:50:45.612 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None
2023-07-28T09:50:45.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-28T09:51:04.052 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3
2023-07-28T09:51:10.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-28T09:51:18.128 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3
2023-07-28T09:51:24.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-28T09:51:32.208 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3
2023-07-28T09:51:38.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-28T09:52:33.392 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2023-07-28T09:52:33.394 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi033
2023-07-28T09:52:38.216 WARNING:teuthology.provision.fog.smithi003:timed out
2023-07-28T09:52:42.400 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com'
2023-07-28T09:52:42.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-28T09:52:47.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-28T09:52:52.733 DEBUG:teuthology.orchestra.run.smithi033:> true
2023-07-28T09:52:52.742 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-28T09:52:52.742 DEBUG:teuthology.orchestra.run.smithi003:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-28T09:52:53.069 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com'
2023-07-28T09:52:53.069 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-28T09:52:53.069 DEBUG:teuthology.orchestra.run.smithi033:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-28T09:52:53.072 INFO:teuthology.provision.fog.smithi003:Node is ready
2023-07-28T09:52:53.072 DEBUG:teuthology.orchestra.run.smithi003:> hostname
2023-07-28T09:52:53.087 INFO:teuthology.provision.fog.smithi033:Node is ready
2023-07-28T09:52:53.087 DEBUG:teuthology.orchestra.run.smithi033:> hostname
2023-07-28T09:52:53.091 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003.front.sepia.ceph.com
2023-07-28T09:52:53.092 DEBUG:teuthology.orchestra.run.smithi003:> grep smithi003.front.sepia.ceph.com /etc/hosts
2023-07-28T09:52:53.145 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033.front.sepia.ceph.com
2023-07-28T09:52:53.146 DEBUG:teuthology.orchestra.run.smithi033:> grep smithi033.front.sepia.ceph.com /etc/hosts
2023-07-28T09:52:53.150 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3 smithi003.front.sepia.ceph.com smithi003
2023-07-28T09:52:53.150 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname smithi003
2023-07-28T09:52:53.204 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33 smithi033.front.sepia.ceph.com smithi033
2023-07-28T09:52:53.204 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname smithi033
2023-07-28T09:52:53.271 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/smithi003.front.sepia.ceph.com/smithi003/g' /etc/hosts
2023-07-28T09:52:53.301 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/172.21.15.3/172.21.15.3/g' /etc/hosts
2023-07-28T09:52:53.319 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/smithi033.front.sepia.ceph.com/smithi033/g' /etc/hosts
2023-07-28T09:52:53.350 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/172.21.15.33/172.21.15.33/g' /etc/hosts
2023-07-28T09:52:53.370 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/smithi003.front.sepia.ceph.com/smithi003/g' /etc/hostname
2023-07-28T09:52:53.418 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/smithi033.front.sepia.ceph.com/smithi033/g' /etc/hostname
2023-07-28T09:52:53.440 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname smithi003
2023-07-28T09:52:53.486 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname smithi033
2023-07-28T09:52:53.508 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release
2023-07-28T09:52:53.553 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release
2023-07-28T09:52:53.565 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="Red Hat Enterprise Linux"
2023-07-28T09:52:53.565 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="8.6 (Ootpa)"
2023-07-28T09:52:53.566 INFO:teuthology.orchestra.run.smithi003.stdout:ID="rhel"
2023-07-28T09:52:53.566 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="fedora"
2023-07-28T09:52:53.566 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="8.6"
2023-07-28T09:52:53.566 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el8"
2023-07-28T09:52:53.566 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-07-28T09:52:53.566 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31"
2023-07-28T09:52:53.566 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-07-28T09:52:53.567 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://www.redhat.com/"
2023-07-28T09:52:53.567 INFO:teuthology.orchestra.run.smithi003.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-07-28T09:52:53.567 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-28T09:52:53.567 INFO:teuthology.orchestra.run.smithi003.stdout:
2023-07-28T09:52:53.567 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-07-28T09:52:53.567 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-07-28T09:52:53.567 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-07-28T09:52:53.568 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-07-28T09:52:53.568 INFO:teuthology.provision.fog.smithi003:Deploy complete!
2023-07-28T09:52:53.569 DEBUG:teuthology.parallel:result is None
2023-07-28T09:52:53.610 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="Red Hat Enterprise Linux"
2023-07-28T09:52:53.610 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="8.6 (Ootpa)"
2023-07-28T09:52:53.610 INFO:teuthology.orchestra.run.smithi033.stdout:ID="rhel"
2023-07-28T09:52:53.610 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="fedora"
2023-07-28T09:52:53.611 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="8.6"
2023-07-28T09:52:53.611 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el8"
2023-07-28T09:52:53.611 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-07-28T09:52:53.611 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31"
2023-07-28T09:52:53.611 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-07-28T09:52:53.611 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://www.redhat.com/"
2023-07-28T09:52:53.611 INFO:teuthology.orchestra.run.smithi033.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-07-28T09:52:53.612 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-28T09:52:53.612 INFO:teuthology.orchestra.run.smithi033.stdout:
2023-07-28T09:52:53.612 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-07-28T09:52:53.612 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-07-28T09:52:53.612 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-07-28T09:52:53.612 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-07-28T09:52:53.612 INFO:teuthology.provision.fog.smithi033:Deploy complete!
2023-07-28T09:52:53.613 DEBUG:teuthology.parallel:result is None
2023-07-28T09:53:13.054 INFO:teuthology.lock.ops:Checking smithi003.front.sepia.ceph.com
2023-07-28T09:53:13.055 INFO:teuthology.lock.ops:New key found. Updating...
2023-07-28T09:53:13.070 INFO:teuthology.lock.ops:Checking smithi033.front.sepia.ceph.com
2023-07-28T09:53:13.072 INFO:teuthology.lock.ops:Updating [smithi003.front.sepia.ceph.com]: set os type and version on server
2023-07-28T09:53:13.073 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com'
2023-07-28T09:53:13.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-28T09:53:18.143 DEBUG:teuthology.orchestra.run.smithi003:> true
2023-07-28T09:53:18.277 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com'
2023-07-28T09:53:18.278 DEBUG:teuthology.orchestra.run.smithi003:> uname -m
2023-07-28T09:53:18.335 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64
2023-07-28T09:53:18.336 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release
2023-07-28T09:53:18.393 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="Red Hat Enterprise Linux"
2023-07-28T09:53:18.394 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="8.6 (Ootpa)"
2023-07-28T09:53:18.394 INFO:teuthology.orchestra.run.smithi003.stdout:ID="rhel"
2023-07-28T09:53:18.394 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="fedora"
2023-07-28T09:53:18.394 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="8.6"
2023-07-28T09:53:18.394 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el8"
2023-07-28T09:53:18.394 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-07-28T09:53:18.395 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31"
2023-07-28T09:53:18.395 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-07-28T09:53:18.395 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://www.redhat.com/"
2023-07-28T09:53:18.395 INFO:teuthology.orchestra.run.smithi003.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-07-28T09:53:18.395 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-28T09:53:18.395 INFO:teuthology.orchestra.run.smithi003.stdout:
2023-07-28T09:53:18.395 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-07-28T09:53:18.395 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-07-28T09:53:18.396 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-07-28T09:53:18.396 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-07-28T09:53:18.396 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server
2023-07-28T09:53:23.428 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: set os type and version on server
2023-07-28T09:53:23.428 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com'
2023-07-28T09:53:23.429 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-28T09:53:28.500 DEBUG:teuthology.orchestra.run.smithi033:> true
2023-07-28T09:53:28.593 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com'
2023-07-28T09:53:28.594 DEBUG:teuthology.orchestra.run.smithi033:> uname -m
2023-07-28T09:53:28.651 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64
2023-07-28T09:53:28.651 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release
2023-07-28T09:53:28.708 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="Red Hat Enterprise Linux"
2023-07-28T09:53:28.709 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="8.6 (Ootpa)"
2023-07-28T09:53:28.709 INFO:teuthology.orchestra.run.smithi033.stdout:ID="rhel"
2023-07-28T09:53:28.709 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="fedora"
2023-07-28T09:53:28.709 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="8.6"
2023-07-28T09:53:28.709 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el8"
2023-07-28T09:53:28.709 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-07-28T09:53:28.709 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31"
2023-07-28T09:53:28.710 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-07-28T09:53:28.710 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://www.redhat.com/"
2023-07-28T09:53:28.710 INFO:teuthology.orchestra.run.smithi033.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-07-28T09:53:28.710 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-28T09:53:28.710 INFO:teuthology.orchestra.run.smithi033.stdout:
2023-07-28T09:53:28.710 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-07-28T09:53:28.710 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-07-28T09:53:28.710 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-07-28T09:53:28.711 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-07-28T09:53:28.711 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server
2023-07-28T09:53:28.735 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T09:53:28.736 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T09:53:28.834 INFO:teuthology.dispatcher.supervisor:Running job 7355042
2023-07-28T09:53:28.834 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-07-28_06:48:51-fs-wip-vshankar-testing-20230727.111459-testing-default-smithi/7355042 --name vshankar-2023-07-28_06:48:51-fs-wip-vshankar-testing-20230727.111459-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} -- /home/teuthworker/archive/vshankar-2023-07-28_06:48:51-fs-wip-vshankar-testing-20230727.111459-testing-default-smithi/7355042/orig.config.yaml
2023-07-28T09:53:28.837 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2023-07-28_06:48:51-fs-wip-vshankar-testing-20230727.111459-testing-default-smithi/7355042
2023-07-28T09:53:28.838 INFO:teuthology.dispatcher.supervisor:Job PID: 25097
2023-07-28T09:53:28.839 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-07-28T09:55:28.936 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T09:55:28.937 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T09:57:31.976 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T09:57:31.977 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T09:59:32.186 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T09:59:32.212 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T10:01:32.409 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T10:01:32.409 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T10:03:32.589 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T10:03:32.589 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T10:05:32.761 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T10:05:32.762 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T10:07:32.921 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T10:07:32.922 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T10:09:33.089 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T10:09:33.108 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T10:11:33.285 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T10:11:33.286 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T10:13:33.457 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T10:13:33.457 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T10:15:33.615 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T10:15:33.616 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T10:17:33.780 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T10:17:33.781 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T10:19:33.938 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T10:19:33.939 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T10:21:34.113 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T10:21:34.114 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T10:23:34.274 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T10:23:34.275 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T10:25:34.438 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T10:25:34.439 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T10:27:34.613 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T10:27:34.614 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T10:29:34.699 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-28T10:29:34.700 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-28T10:29:34.764 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-07-28T10:29:34.951 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-07-28T10:29:34.952 INFO:teuthology.nuke:Checking targets against current locks
2023-07-28T10:29:35.007 DEBUG:teuthology.nuke:shortname: smithi003
2023-07-28T10:29:35.008 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-07-28T10:29:35.035 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-07-28_06:48:51-fs-wip-vshankar-testing-20230727.111459-testing-default-smithi/7355042', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-28 09:46:26.104818', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}
2023-07-28T10:29:35.038 DEBUG:teuthology.nuke:shortname: smithi033
2023-07-28T10:29:35.038 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-07-28T10:29:35.061 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-07-28_06:48:51-fs-wip-vshankar-testing-20230727.111459-testing-default-smithi/7355042', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-28 09:46:26.102719', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}
2023-07-28T10:29:35.097 INFO:teuthology.orchestra.console.smithi003:Power off
2023-07-28T10:29:35.097 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-28T10:29:35.120 DEBUG:teuthology.orchestra.console.smithi003:power off output: Chassis Power Control: Down/Off
2023-07-28T10:29:35.120 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T10:29:35.136 INFO:teuthology.orchestra.console.smithi033:Power off
2023-07-28T10:29:35.137 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-28T10:29:35.147 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off
2023-07-28T10:29:35.159 DEBUG:teuthology.orchestra.console.smithi033:power off output: Chassis Power Control: Down/Off
2023-07-28T10:29:35.159 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-28T10:29:35.182 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off
2023-07-28T10:29:35.249 INFO:teuthology.orchestra.console.smithi003:Power off completed
2023-07-28T10:29:35.284 INFO:teuthology.orchestra.console.smithi033:Power off completed
2023-07-28T10:29:35.413 INFO:teuthology.lock.ops:unlocked: smithi003.front.sepia.ceph.com
2023-07-28T10:29:35.429 INFO:teuthology.lock.ops:unlocked: smithi033.front.sepia.ceph.com