2023-09-26T17:04:52.869 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-26T17:04:52.869 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:04:53.036 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0d6cdc6670>, signals=[15]) 2023-09-26T17:04:53.036 INFO:teuthology.lock.ops:Start node 'smithi092.front.sepia.ceph.com' reimaging 2023-09-26T17:04:53.036 INFO:teuthology.lock.ops:Updating [smithi092.front.sepia.ceph.com]: reset os type and version on server 2023-09-26T17:04:53.037 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2023-09-26T17:04:53.059 INFO:teuthology.lock.ops:Start node 'smithi099.front.sepia.ceph.com' reimaging 2023-09-26T17:04:53.059 INFO:teuthology.lock.ops:Updating [smithi099.front.sepia.ceph.com]: reset os type and version on server 2023-09-26T17:04:53.059 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2023-09-26T17:04:53.060 DEBUG:teuthology.provision.fog.smithi092:FOG request: GET /host {"name": "smithi092"} 2023-09-26T17:04:53.083 INFO:teuthology.lock.ops:Start node 'smithi118.front.sepia.ceph.com' reimaging 2023-09-26T17:04:53.084 INFO:teuthology.lock.ops:Updating [smithi118.front.sepia.ceph.com]: reset os type and version on server 2023-09-26T17:04:53.084 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2023-09-26T17:04:53.085 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /host {"name": "smithi099"} 2023-09-26T17:04:53.108 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /host {"name": "smithi118"} 2023-09-26T17:04:53.142 DEBUG:teuthology.provision.fog.smithi092:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-09-26T17:04:53.163 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-09-26T17:04:53.172 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-09-26T17:04:53.184 DEBUG:teuthology.provision.fog.smithi092:Requesting image smithi_centos_8.stream (ID 3) 2023-09-26T17:04:53.184 DEBUG:teuthology.provision.fog.smithi092:FOG request: PUT /host/197 {"imageID": 3} 2023-09-26T17:04:53.202 DEBUG:teuthology.provision.fog.smithi099:Requesting image smithi_centos_8.stream (ID 3) 2023-09-26T17:04:53.202 DEBUG:teuthology.provision.fog.smithi099:FOG request: PUT /host/204 {"imageID": 3} 2023-09-26T17:04:53.220 DEBUG:teuthology.provision.fog.smithi118:Requesting image smithi_centos_8.stream (ID 3) 2023-09-26T17:04:53.220 DEBUG:teuthology.provision.fog.smithi118:FOG request: PUT /host/223 {"imageID": 3} 2023-09-26T17:04:53.224 INFO:teuthology.provision.fog.smithi092:Scheduling deploy of centos 8.stream 2023-09-26T17:04:53.224 DEBUG:teuthology.provision.fog.smithi092:FOG request: GET /task/active None 2023-09-26T17:04:53.246 INFO:teuthology.provision.fog.smithi099:Scheduling deploy of centos 8.stream 2023-09-26T17:04:53.246 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-09-26T17:04:53.261 INFO:teuthology.provision.fog.smithi118:Scheduling deploy of centos 8.stream 2023-09-26T17:04:53.261 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-26T17:04:53.283 DEBUG:teuthology.provision.fog.smithi092:FOG request: GET /tasktype {"name": "deploy"} 2023-09-26T17:04:53.300 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /tasktype {"name": "deploy"} 2023-09-26T17:04:53.317 DEBUG:teuthology.provision.fog.smithi092:FOG request: POST /host/197/task {"taskTypeID": 1} 2023-09-26T17:04:53.321 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /tasktype {"name": "deploy"} 2023-09-26T17:04:53.337 DEBUG:teuthology.provision.fog.smithi099:FOG request: POST /host/204/task {"taskTypeID": 1} 2023-09-26T17:04:53.351 DEBUG:teuthology.provision.fog.smithi118:FOG request: POST /host/223/task {"taskTypeID": 1} 2023-09-26T17:04:53.380 DEBUG:teuthology.provision.fog.smithi092:FOG request: GET /task/active None 2023-09-26T17:04:53.389 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-09-26T17:04:53.404 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-26T17:04:53.483 INFO:teuthology.orchestra.console.smithi092:Power off 2023-09-26T17:04:53.483 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-26T17:04:53.495 INFO:teuthology.orchestra.console.smithi118:Power off 2023-09-26T17:04:53.495 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-26T17:04:53.504 INFO:teuthology.orchestra.console.smithi099:Power off 2023-09-26T17:04:53.504 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-26T17:04:53.512 DEBUG:teuthology.orchestra.console.smithi092:power off output: Chassis Power Control: Down/Off 2023-09-26T17:04:53.512 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:04:53.520 DEBUG:teuthology.orchestra.console.smithi118:power off output: Chassis Power Control: Down/Off 2023-09-26T17:04:53.520 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:04:53.527 DEBUG:teuthology.orchestra.console.smithi099:power off output: Chassis Power Control: Down/Off 2023-09-26T17:04:53.528 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:04:53.535 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2023-09-26T17:04:53.544 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2023-09-26T17:04:53.547 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2023-09-26T17:04:57.538 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:04:57.545 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:04:57.550 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:04:57.646 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2023-09-26T17:04:57.653 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2023-09-26T17:04:57.658 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2023-09-26T17:05:01.649 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:05:01.656 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:05:01.662 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:05:01.758 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2023-09-26T17:05:01.763 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2023-09-26T17:05:01.769 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off 2023-09-26T17:05:01.864 INFO:teuthology.orchestra.console.smithi118:Power off completed 2023-09-26T17:05:01.870 INFO:teuthology.orchestra.console.smithi099:Power off completed 2023-09-26T17:05:01.965 INFO:teuthology.orchestra.console.smithi118:Power on 2023-09-26T17:05:01.965 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-26T17:05:01.971 INFO:teuthology.orchestra.console.smithi099:Power on 2023-09-26T17:05:01.972 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-26T17:05:01.984 DEBUG:teuthology.orchestra.console.smithi118:power on output: Chassis Power Control: Up/On 2023-09-26T17:05:01.984 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:05:01.993 DEBUG:teuthology.orchestra.console.smithi099:power on output: Chassis Power Control: Up/On 2023-09-26T17:05:01.994 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:05:02.007 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2023-09-26T17:05:02.013 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off 2023-09-26T17:05:05.762 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:05:05.870 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2023-09-26T17:05:05.971 INFO:teuthology.orchestra.console.smithi092:Power off completed 2023-09-26T17:05:06.007 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:05:06.013 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:05:06.072 INFO:teuthology.orchestra.console.smithi092:Power on 2023-09-26T17:05:06.072 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-26T17:05:06.093 DEBUG:teuthology.orchestra.console.smithi092:power on output: Chassis Power Control: Up/On 2023-09-26T17:05:06.093 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:05:06.112 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2023-09-26T17:05:06.115 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2023-09-26T17:05:06.121 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off 2023-09-26T17:05:10.118 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:05:10.125 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:05:10.131 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T17:05:10.228 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2023-09-26T17:05:10.233 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2023-09-26T17:05:10.240 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2023-09-26T17:05:10.329 INFO:teuthology.orchestra.console.smithi092:Power on completed 2023-09-26T17:05:10.334 INFO:teuthology.orchestra.console.smithi118:Power on completed 2023-09-26T17:05:10.341 INFO:teuthology.orchestra.console.smithi099:Power on completed 2023-09-26T17:05:10.430 INFO:teuthology.provision.fog.smithi092:Waiting for deploy to finish 2023-09-26T17:05:10.431 DEBUG:teuthology.provision.fog.smithi092:FOG request: GET /task/active None 2023-09-26T17:05:10.434 INFO:teuthology.provision.fog.smithi118:Waiting for deploy to finish 2023-09-26T17:05:10.435 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-26T17:05:10.443 INFO:teuthology.provision.fog.smithi099:Waiting for deploy to finish 2023-09-26T17:05:10.443 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-09-26T17:05:25.520 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-26T17:05:25.522 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-09-26T17:05:25.523 DEBUG:teuthology.provision.fog.smithi092:FOG request: GET /task/active None 2023-09-26T17:05:40.600 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-09-26T17:05:40.602 DEBUG:teuthology.provision.fog.smithi092:FOG request: GET /task/active None 2023-09-26T17:05:40.602 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-26T17:05:55.675 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-26T17:05:55.677 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-09-26T17:05:55.678 DEBUG:teuthology.provision.fog.smithi092:FOG request: GET /task/active None 2023-09-26T17:06:10.746 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-26T17:06:10.747 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-09-26T17:06:10.748 DEBUG:teuthology.provision.fog.smithi092:FOG request: GET /task/active None 2023-09-26T17:06:25.822 DEBUG:teuthology.provision.fog.smithi092:FOG request: GET /task/active None 2023-09-26T17:06:25.824 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-09-26T17:06:25.825 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-26T17:06:40.890 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-26T17:06:40.892 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-09-26T17:06:40.893 DEBUG:teuthology.provision.fog.smithi092:FOG request: GET /task/active None 2023-09-26T17:06:55.956 DEBUG:teuthology.provision.fog.smithi092:FOG request: GET /task/active None 2023-09-26T17:06:55.959 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-26T17:06:55.963 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-09-26T17:07:11.035 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-09-26T17:07:11.037 DEBUG:teuthology.provision.fog.smithi092:FOG request: GET /task/active None 2023-09-26T17:07:11.038 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-26T17:07:26.102 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-09-26T17:07:26.104 DEBUG:teuthology.provision.fog.smithi092:FOG request: GET /task/active None 2023-09-26T17:07:26.105 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-26T17:07:41.170 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-26T17:07:41.172 DEBUG:teuthology.provision.fog.smithi092:FOG request: GET /task/active None 2023-09-26T17:07:41.172 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-09-26T17:07:56.239 DEBUG:teuthology.provision.fog.smithi099:FOG request: GET /task/active None 2023-09-26T17:07:56.242 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-26T17:07:56.243 DEBUG:teuthology.provision.fog.smithi092:FOG request: GET /task/active None 2023-09-26T17:07:56.298 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:07:56.302 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:08:11.315 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-26T17:08:14.842 WARNING:teuthology.provision.fog.smithi092:[Errno None] Unable to connect to port 22 on 172.21.15.92 2023-09-26T17:08:14.843 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99 2023-09-26T17:08:20.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:08:20.849 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:08:23.902 WARNING:teuthology.provision.fog.smithi092:[Errno None] Unable to connect to port 22 on 172.21.15.92 2023-09-26T17:08:23.930 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99 2023-09-26T17:08:26.373 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-26T17:08:26.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:08:29.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:08:29.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:08:32.986 WARNING:teuthology.provision.fog.smithi092:[Errno None] Unable to connect to port 22 on 172.21.15.92 2023-09-26T17:08:32.987 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99 2023-09-26T17:08:38.994 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:08:38.996 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:08:42.075 WARNING:teuthology.provision.fog.smithi092:[Errno None] Unable to connect to port 22 on 172.21.15.92 2023-09-26T17:08:42.075 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99 2023-09-26T17:08:44.794 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-09-26T17:08:48.079 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:08:48.081 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:08:50.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:08:53.850 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-09-26T17:08:59.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:09:02.913 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-09-26T17:09:08.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:09:11.998 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-09-26T17:09:18.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:09:21.082 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-09-26T17:09:27.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:09:48.101 WARNING:teuthology.provision.fog.smithi092:timed out 2023-09-26T17:09:48.101 WARNING:teuthology.provision.fog.smithi099:timed out 2023-09-26T17:09:54.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:09:54.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:09:54.332 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-26T17:09:54.332 DEBUG:teuthology.orchestra.run.smithi092:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-26T17:09:54.431 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-26T17:09:54.432 DEBUG:teuthology.orchestra.run.smithi099:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-26T17:09:54.582 INFO:teuthology.provision.fog.smithi092:Node is ready 2023-09-26T17:09:54.582 DEBUG:teuthology.orchestra.run.smithi092:> hostname 2023-09-26T17:09:54.604 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092.front.sepia.ceph.com 2023-09-26T17:09:54.604 DEBUG:teuthology.orchestra.run.smithi092:> grep smithi092.front.sepia.ceph.com /etc/hosts 2023-09-26T17:09:54.658 INFO:teuthology.provision.fog.smithi099:Node is ready 2023-09-26T17:09:54.659 DEBUG:teuthology.orchestra.run.smithi099:> hostname 2023-09-26T17:09:54.665 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92 smithi092.front.sepia.ceph.com smithi092 2023-09-26T17:09:54.666 DEBUG:teuthology.orchestra.run.smithi092:> sudo hostname smithi092 2023-09-26T17:09:54.718 INFO:teuthology.orchestra.run.smithi099.stdout:smithi099.front.sepia.ceph.com 2023-09-26T17:09:54.719 DEBUG:teuthology.orchestra.run.smithi099:> grep smithi099.front.sepia.ceph.com /etc/hosts 2023-09-26T17:09:54.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i -e 's/smithi092.front.sepia.ceph.com/smithi092/g' /etc/hosts 2023-09-26T17:09:54.778 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99 smithi099.front.sepia.ceph.com smithi099 2023-09-26T17:09:54.778 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname smithi099 2023-09-26T17:09:54.800 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i -e 's/172.21.15.92/172.21.15.92/g' /etc/hosts 2023-09-26T17:09:54.872 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i -e 's/smithi092.front.sepia.ceph.com/smithi092/g' /etc/hostname 2023-09-26T17:09:54.889 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/smithi099.front.sepia.ceph.com/smithi099/g' /etc/hosts 2023-09-26T17:09:54.918 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/172.21.15.99/172.21.15.99/g' /etc/hosts 2023-09-26T17:09:54.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo hostname smithi092 2023-09-26T17:09:54.985 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/smithi099.front.sepia.ceph.com/smithi099/g' /etc/hostname 2023-09-26T17:09:55.012 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2023-09-26T17:09:55.053 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname smithi099 2023-09-26T17:09:55.071 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2023-09-26T17:09:55.071 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="8" 2023-09-26T17:09:55.072 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2023-09-26T17:09:55.072 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2023-09-26T17:09:55.072 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="8" 2023-09-26T17:09:55.072 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el8" 2023-09-26T17:09:55.072 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-26T17:09:55.072 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2023-09-26T17:09:55.072 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-26T17:09:55.072 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2023-09-26T17:09:55.072 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-26T17:09:55.072 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-26T17:09:55.072 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-26T17:09:55.073 INFO:teuthology.provision.fog.smithi092:Deploy complete! 2023-09-26T17:09:55.074 DEBUG:teuthology.parallel:result is None 2023-09-26T17:09:55.120 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2023-09-26T17:09:55.177 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2023-09-26T17:09:55.178 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="8" 2023-09-26T17:09:55.178 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2023-09-26T17:09:55.178 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2023-09-26T17:09:55.178 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="8" 2023-09-26T17:09:55.178 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el8" 2023-09-26T17:09:55.178 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-26T17:09:55.178 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2023-09-26T17:09:55.178 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-26T17:09:55.178 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2023-09-26T17:09:55.178 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-26T17:09:55.179 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-26T17:09:55.179 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-26T17:09:55.179 INFO:teuthology.provision.fog.smithi099:Deploy complete! 2023-09-26T17:09:55.179 DEBUG:teuthology.parallel:result is None 2023-09-26T17:10:27.113 WARNING:teuthology.provision.fog.smithi118:timed out 2023-09-26T17:10:33.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:10:33.355 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-26T17:10:33.355 DEBUG:teuthology.orchestra.run.smithi118:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-26T17:10:33.598 INFO:teuthology.provision.fog.smithi118:Node is ready 2023-09-26T17:10:33.598 DEBUG:teuthology.orchestra.run.smithi118:> hostname 2023-09-26T17:10:33.619 INFO:teuthology.orchestra.run.smithi118.stdout:smithi118.front.sepia.ceph.com 2023-09-26T17:10:33.620 DEBUG:teuthology.orchestra.run.smithi118:> grep smithi118.front.sepia.ceph.com /etc/hosts 2023-09-26T17:10:33.682 INFO:teuthology.orchestra.run.smithi118.stdout:172.21.15.118 smithi118.front.sepia.ceph.com smithi118 2023-09-26T17:10:33.682 DEBUG:teuthology.orchestra.run.smithi118:> sudo hostname smithi118 2023-09-26T17:10:33.793 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/smithi118.front.sepia.ceph.com/smithi118/g' /etc/hosts 2023-09-26T17:10:33.825 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/172.21.15.118/172.21.15.118/g' /etc/hosts 2023-09-26T17:10:33.895 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/smithi118.front.sepia.ceph.com/smithi118/g' /etc/hostname 2023-09-26T17:10:33.965 DEBUG:teuthology.orchestra.run.smithi118:> sudo hostname smithi118 2023-09-26T17:10:34.033 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2023-09-26T17:10:34.091 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="CentOS Stream" 2023-09-26T17:10:34.091 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="8" 2023-09-26T17:10:34.091 INFO:teuthology.orchestra.run.smithi118.stdout:ID="centos" 2023-09-26T17:10:34.091 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE="rhel fedora" 2023-09-26T17:10:34.091 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="8" 2023-09-26T17:10:34.091 INFO:teuthology.orchestra.run.smithi118.stdout:PLATFORM_ID="platform:el8" 2023-09-26T17:10:34.092 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-26T17:10:34.092 INFO:teuthology.orchestra.run.smithi118.stdout:ANSI_COLOR="0;31" 2023-09-26T17:10:34.092 INFO:teuthology.orchestra.run.smithi118.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-26T17:10:34.092 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://centos.org/" 2023-09-26T17:10:34.092 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-26T17:10:34.092 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-26T17:10:34.092 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-26T17:10:34.092 INFO:teuthology.provision.fog.smithi118:Deploy complete! 2023-09-26T17:10:34.093 DEBUG:teuthology.parallel:result is None 2023-09-26T17:10:34.303 INFO:teuthology.lock.ops:Checking smithi092.front.sepia.ceph.com 2023-09-26T17:10:34.303 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-26T17:10:34.329 INFO:teuthology.lock.ops:Checking smithi099.front.sepia.ceph.com 2023-09-26T17:10:34.330 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-26T17:10:34.354 INFO:teuthology.lock.ops:Checking smithi118.front.sepia.ceph.com 2023-09-26T17:10:34.354 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-26T17:10:34.372 INFO:teuthology.lock.ops:Updating [smithi092.front.sepia.ceph.com]: set os type and version on server 2023-09-26T17:10:34.372 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-09-26T17:10:34.372 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:10:34.439 DEBUG:teuthology.orchestra.run.smithi092:> true 2023-09-26T17:10:34.534 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-09-26T17:10:34.534 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2023-09-26T17:10:34.593 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2023-09-26T17:10:34.593 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2023-09-26T17:10:34.652 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2023-09-26T17:10:34.652 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="8" 2023-09-26T17:10:34.652 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2023-09-26T17:10:34.652 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2023-09-26T17:10:34.652 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="8" 2023-09-26T17:10:34.653 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el8" 2023-09-26T17:10:34.653 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-26T17:10:34.653 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2023-09-26T17:10:34.653 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-26T17:10:34.653 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2023-09-26T17:10:34.653 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-26T17:10:34.653 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-26T17:10:34.653 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-26T17:10:34.654 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2023-09-26T17:10:34.679 INFO:teuthology.lock.ops:Updating [smithi099.front.sepia.ceph.com]: set os type and version on server 2023-09-26T17:10:34.680 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-09-26T17:10:34.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:10:34.746 DEBUG:teuthology.orchestra.run.smithi099:> true 2023-09-26T17:10:34.838 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2023-09-26T17:10:34.838 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2023-09-26T17:10:34.896 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2023-09-26T17:10:34.897 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2023-09-26T17:10:34.955 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2023-09-26T17:10:34.956 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="8" 2023-09-26T17:10:34.956 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2023-09-26T17:10:34.956 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2023-09-26T17:10:34.956 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="8" 2023-09-26T17:10:34.956 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el8" 2023-09-26T17:10:34.956 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-26T17:10:34.956 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2023-09-26T17:10:34.956 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-26T17:10:34.956 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2023-09-26T17:10:34.956 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-26T17:10:34.957 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-26T17:10:34.957 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-26T17:10:34.957 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2023-09-26T17:10:34.979 INFO:teuthology.lock.ops:Updating [smithi118.front.sepia.ceph.com]: set os type and version on server 2023-09-26T17:10:34.979 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2023-09-26T17:10:34.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-26T17:10:35.047 DEBUG:teuthology.orchestra.run.smithi118:> true 2023-09-26T17:10:35.121 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2023-09-26T17:10:35.121 DEBUG:teuthology.orchestra.run.smithi118:> uname -m 2023-09-26T17:10:35.183 INFO:teuthology.orchestra.run.smithi118.stdout:x86_64 2023-09-26T17:10:35.183 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2023-09-26T17:10:35.245 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="CentOS Stream" 2023-09-26T17:10:35.245 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="8" 2023-09-26T17:10:35.245 INFO:teuthology.orchestra.run.smithi118.stdout:ID="centos" 2023-09-26T17:10:35.245 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE="rhel fedora" 2023-09-26T17:10:35.245 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="8" 2023-09-26T17:10:35.245 INFO:teuthology.orchestra.run.smithi118.stdout:PLATFORM_ID="platform:el8" 2023-09-26T17:10:35.245 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-26T17:10:35.245 INFO:teuthology.orchestra.run.smithi118.stdout:ANSI_COLOR="0;31" 2023-09-26T17:10:35.245 INFO:teuthology.orchestra.run.smithi118.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-26T17:10:35.246 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://centos.org/" 2023-09-26T17:10:35.246 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-26T17:10:35.246 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-26T17:10:35.246 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-26T17:10:35.246 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2023-09-26T17:10:35.271 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:10:35.371 INFO:teuthology.dispatcher.supervisor:Running job 7402405 2023-09-26T17:10:35.371 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-09-26_14:09:41-fs-wip-vshankar-testing-20230926.081818-testing-default-smithi/7402405 --name vshankar-2023-09-26_14:09:41-fs-wip-vshankar-testing-20230926.081818-testing-default-smithi --description fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}} -- /home/teuthworker/archive/vshankar-2023-09-26_14:09:41-fs-wip-vshankar-testing-20230926.081818-testing-default-smithi/7402405/orig.config.yaml 2023-09-26T17:10:35.374 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2023-09-26_14:09:41-fs-wip-vshankar-testing-20230926.081818-testing-default-smithi/7402405 2023-09-26T17:10:35.375 INFO:teuthology.dispatcher.supervisor:Job PID: 826956 2023-09-26T17:10:35.375 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-09-26T17:12:35.475 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:14:35.647 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:16:35.761 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:18:35.931 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:20:36.091 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:22:36.284 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:24:36.455 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:26:36.627 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:28:36.801 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:30:36.958 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:32:37.127 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:34:37.300 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:36:37.471 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:38:37.635 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:40:37.746 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:42:37.885 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:44:38.059 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:46:38.226 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:48:38.370 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:50:38.712 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:52:38.871 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:54:38.994 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:56:39.167 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T17:58:39.417 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:00:39.497 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:02:39.667 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:04:39.831 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:06:40.033 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:08:40.158 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:10:40.248 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:12:40.405 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:14:40.575 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:16:40.651 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:18:40.769 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:20:40.938 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:22:41.161 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:24:41.300 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:26:41.430 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:28:41.595 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:30:41.855 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:32:42.020 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:34:42.180 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:36:42.370 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:38:42.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-26T18:38:42.615 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-09-26T18:38:42.827 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-09-26T18:38:42.827 INFO:teuthology.nuke:Checking targets against current locks 2023-09-26T18:38:42.876 DEBUG:teuthology.nuke:shortname: smithi092 2023-09-26T18:38:42.876 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-26T18:38:42.903 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-26_14:09:41-fs-wip-vshankar-testing-20230926.081818-testing-default-smithi/7402405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-26 17:04:52.064052', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGKWM1NWy87LsbF93dZv9tVeZaXKQO6nzPu8lQbeSUdWuT3mTqgNrrvlamCs0W54ZN6MNyWoNLTzsLH6yNfcP20='} 2023-09-26T18:38:42.907 DEBUG:teuthology.nuke:shortname: smithi099 2023-09-26T18:38:42.907 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-26T18:38:42.931 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-26_14:09:41-fs-wip-vshankar-testing-20230926.081818-testing-default-smithi/7402405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-26 17:04:52.067574', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC4yO2JT7NtbBdozibbYyzPoA21elkrX54tq7tnCgNMSthstSaldZdQN/WCjeKLY77aLtCW4nHI3856fUW/eQRM='} 2023-09-26T18:38:42.935 DEBUG:teuthology.nuke:shortname: smithi118 2023-09-26T18:38:42.935 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-26T18:38:42.968 INFO:teuthology.orchestra.console.smithi092:Power off 2023-09-26T18:38:42.968 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-26T18:38:42.975 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi118.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-26_14:09:41-fs-wip-vshankar-testing-20230926.081818-testing-default-smithi/7402405', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-26 17:04:52.066052', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOspnzRamh3U76qevLlUmbZ4f1QwcaMWkXMY2/6swoN3cgtyYUgFprrwgPN6d81g/jevtn86qsWrwetM53LIrrY='} 2023-09-26T18:38:42.989 DEBUG:teuthology.orchestra.console.smithi092:power off output: Chassis Power Control: Down/Off 2023-09-26T18:38:42.989 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T18:38:43.005 INFO:teuthology.orchestra.console.smithi099:Power off 2023-09-26T18:38:43.005 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-26T18:38:43.013 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2023-09-26T18:38:43.025 DEBUG:teuthology.orchestra.console.smithi099:power off output: Chassis Power Control: Down/Off 2023-09-26T18:38:43.025 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T18:38:43.040 INFO:teuthology.orchestra.console.smithi118:Power off 2023-09-26T18:38:43.040 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-26T18:38:43.048 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off 2023-09-26T18:38:43.061 DEBUG:teuthology.orchestra.console.smithi118:power off output: Chassis Power Control: Down/Off 2023-09-26T18:38:43.062 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-26T18:38:43.081 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2023-09-26T18:38:43.114 INFO:teuthology.orchestra.console.smithi092:Power off completed 2023-09-26T18:38:43.149 INFO:teuthology.orchestra.console.smithi099:Power off completed 2023-09-26T18:38:43.182 INFO:teuthology.orchestra.console.smithi118:Power off completed 2023-09-26T18:38:43.273 INFO:teuthology.lock.ops:unlocked: smithi092.front.sepia.ceph.com 2023-09-26T18:38:43.297 INFO:teuthology.lock.ops:unlocked: smithi099.front.sepia.ceph.com 2023-09-26T18:38:43.330 INFO:teuthology.lock.ops:unlocked: smithi118.front.sepia.ceph.com