2023-11-14T06:22:31.906 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-14T06:22:31.909 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:22:32.097 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f70973c6670>, signals=[15]) 2023-11-14T06:22:32.099 INFO:teuthology.lock.ops:Start node 'smithi012.front.sepia.ceph.com' reimaging 2023-11-14T06:22:32.099 INFO:teuthology.lock.ops:Updating [smithi012.front.sepia.ceph.com]: reset os type and version on server 2023-11-14T06:22:32.099 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2023-11-14T06:22:32.127 INFO:teuthology.lock.ops:Start node 'smithi029.front.sepia.ceph.com' reimaging 2023-11-14T06:22:32.127 INFO:teuthology.lock.ops:Updating [smithi029.front.sepia.ceph.com]: reset os type and version on server 2023-11-14T06:22:32.127 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2023-11-14T06:22:32.129 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /host {"name": "smithi012"} 2023-11-14T06:22:32.151 INFO:teuthology.lock.ops:Start node 'smithi148.front.sepia.ceph.com' reimaging 2023-11-14T06:22:32.151 INFO:teuthology.lock.ops:Updating [smithi148.front.sepia.ceph.com]: reset os type and version on server 2023-11-14T06:22:32.152 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2023-11-14T06:22:32.153 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /host {"name": "smithi029"} 2023-11-14T06:22:32.179 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /host {"name": "smithi148"} 2023-11-14T06:22:32.206 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-11-14T06:22:32.221 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-11-14T06:22:32.245 DEBUG:teuthology.provision.fog.smithi012:Requesting image smithi_rhel_8.6 (ID 5) 2023-11-14T06:22:32.246 DEBUG:teuthology.provision.fog.smithi012:FOG request: PUT /host/118 {"imageID": 5} 2023-11-14T06:22:32.249 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-11-14T06:22:32.255 DEBUG:teuthology.provision.fog.smithi029:Requesting image smithi_rhel_8.6 (ID 5) 2023-11-14T06:22:32.255 DEBUG:teuthology.provision.fog.smithi029:FOG request: PUT /host/134 {"imageID": 5} 2023-11-14T06:22:32.298 DEBUG:teuthology.provision.fog.smithi148:Requesting image smithi_rhel_8.6 (ID 5) 2023-11-14T06:22:32.298 DEBUG:teuthology.provision.fog.smithi148:FOG request: PUT /host/253 {"imageID": 5} 2023-11-14T06:22:32.302 INFO:teuthology.provision.fog.smithi012:Scheduling deploy of rhel 8.6 2023-11-14T06:22:32.303 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /task/active None 2023-11-14T06:22:32.312 INFO:teuthology.provision.fog.smithi029:Scheduling deploy of rhel 8.6 2023-11-14T06:22:32.312 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-11-14T06:22:32.346 INFO:teuthology.provision.fog.smithi148:Scheduling deploy of rhel 8.6 2023-11-14T06:22:32.347 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-14T06:22:32.361 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /tasktype {"name": "deploy"} 2023-11-14T06:22:32.368 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /tasktype {"name": "deploy"} 2023-11-14T06:22:32.396 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /tasktype {"name": "deploy"} 2023-11-14T06:22:32.399 DEBUG:teuthology.provision.fog.smithi012:FOG request: POST /host/118/task {"taskTypeID": 1} 2023-11-14T06:22:32.403 DEBUG:teuthology.provision.fog.smithi029:FOG request: POST /host/134/task {"taskTypeID": 1} 2023-11-14T06:22:32.435 DEBUG:teuthology.provision.fog.smithi148:FOG request: POST /host/253/task {"taskTypeID": 1} 2023-11-14T06:22:32.448 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /task/active None 2023-11-14T06:22:32.461 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-11-14T06:22:32.490 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-14T06:22:32.556 INFO:teuthology.orchestra.console.smithi029:Power off 2023-11-14T06:22:32.557 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-14T06:22:32.594 INFO:teuthology.orchestra.console.smithi012:Power off 2023-11-14T06:22:32.595 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-14T06:22:32.612 INFO:teuthology.orchestra.console.smithi148:Power off 2023-11-14T06:22:32.613 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-14T06:22:32.634 DEBUG:teuthology.orchestra.console.smithi029:power off output: Chassis Power Control: Down/Off 2023-11-14T06:22:32.635 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:32.649 DEBUG:teuthology.orchestra.console.smithi012:power off output: Chassis Power Control: Down/Off 2023-11-14T06:22:32.649 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:32.661 DEBUG:teuthology.orchestra.console.smithi148:power off output: Chassis Power Control: Down/Off 2023-11-14T06:22:32.661 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:32.679 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is on 2023-11-14T06:22:32.683 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2023-11-14T06:22:32.694 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-11-14T06:22:36.683 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:36.691 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:36.698 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:36.793 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is on 2023-11-14T06:22:36.799 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2023-11-14T06:22:36.807 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-11-14T06:22:40.798 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:40.806 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:40.820 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:40.909 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is off 2023-11-14T06:22:40.922 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is off 2023-11-14T06:22:40.931 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-11-14T06:22:41.010 INFO:teuthology.orchestra.console.smithi029:Power off completed 2023-11-14T06:22:41.023 INFO:teuthology.orchestra.console.smithi012:Power off completed 2023-11-14T06:22:41.112 INFO:teuthology.orchestra.console.smithi029:Power on 2023-11-14T06:22:41.112 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-14T06:22:41.126 INFO:teuthology.orchestra.console.smithi012:Power on 2023-11-14T06:22:41.127 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-14T06:22:41.139 DEBUG:teuthology.orchestra.console.smithi029:power on output: Chassis Power Control: Up/On 2023-11-14T06:22:41.140 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:41.153 DEBUG:teuthology.orchestra.console.smithi012:power on output: Chassis Power Control: Up/On 2023-11-14T06:22:41.154 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:41.170 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is off 2023-11-14T06:22:41.177 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is off 2023-11-14T06:22:44.936 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:45.045 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2023-11-14T06:22:45.147 INFO:teuthology.orchestra.console.smithi148:Power off completed 2023-11-14T06:22:45.171 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:45.178 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:45.248 INFO:teuthology.orchestra.console.smithi148:Power on 2023-11-14T06:22:45.248 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-14T06:22:45.271 DEBUG:teuthology.orchestra.console.smithi148:power on output: Chassis Power Control: Up/On 2023-11-14T06:22:45.272 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:45.281 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is off 2023-11-14T06:22:45.287 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is off 2023-11-14T06:22:45.296 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2023-11-14T06:22:49.287 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:49.295 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:49.306 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:22:49.408 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2023-11-14T06:22:49.421 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-11-14T06:22:49.511 INFO:teuthology.orchestra.console.smithi012:Power on completed 2023-11-14T06:22:49.523 INFO:teuthology.orchestra.console.smithi148:Power on completed 2023-11-14T06:22:49.524 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is on 2023-11-14T06:22:49.612 INFO:teuthology.provision.fog.smithi012:Waiting for deploy to finish 2023-11-14T06:22:49.613 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /task/active None 2023-11-14T06:22:49.624 INFO:teuthology.provision.fog.smithi148:Waiting for deploy to finish 2023-11-14T06:22:49.624 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-14T06:22:49.627 INFO:teuthology.orchestra.console.smithi029:Power on completed 2023-11-14T06:22:49.729 INFO:teuthology.provision.fog.smithi029:Waiting for deploy to finish 2023-11-14T06:22:49.729 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-11-14T06:23:04.691 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /task/active None 2023-11-14T06:23:04.693 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-14T06:23:04.793 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-11-14T06:23:19.778 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /task/active None 2023-11-14T06:23:19.780 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-14T06:23:19.864 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-11-14T06:23:34.866 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /task/active None 2023-11-14T06:23:34.867 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-14T06:23:34.930 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-11-14T06:23:49.950 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-14T06:23:49.953 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /task/active None 2023-11-14T06:23:49.998 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-11-14T06:24:05.038 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /task/active None 2023-11-14T06:24:05.040 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-14T06:24:05.062 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-11-14T06:24:20.134 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-14T06:24:20.136 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /task/active None 2023-11-14T06:24:20.139 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-11-14T06:24:35.214 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /task/active None 2023-11-14T06:24:35.216 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-14T06:24:35.217 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-11-14T06:24:50.298 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-11-14T06:24:50.300 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /task/active None 2023-11-14T06:24:50.301 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-14T06:25:05.388 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-11-14T06:25:05.393 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /task/active None 2023-11-14T06:25:05.394 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-14T06:25:05.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:25:05.759 ERROR:teuthology.orchestra.connection:Error authenticating with smithi012.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-14T06:25:20.455 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-11-14T06:25:20.459 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-14T06:25:35.526 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-14T06:25:35.537 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-11-14T06:25:35.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:25:38.654 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-14T06:25:44.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:25:47.706 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-14T06:25:50.601 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-11-14T06:25:53.708 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:25:56.766 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-14T06:26:02.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:26:05.685 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-11-14T06:26:05.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:26:05.818 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-14T06:26:08.826 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-11-14T06:26:09.765 WARNING:teuthology.provision.fog.smithi012:timed out 2023-11-14T06:26:11.820 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:26:14.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:26:14.874 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-14T06:26:15.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:26:17.886 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-11-14T06:26:18.842 WARNING:teuthology.provision.fog.smithi012:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-11-14T06:26:20.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:26:23.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:26:23.930 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-14T06:26:24.844 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:26:26.938 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-11-14T06:26:27.898 WARNING:teuthology.provision.fog.smithi012:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-11-14T06:26:29.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:26:32.940 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:26:32.986 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-14T06:26:33.899 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:26:35.998 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-11-14T06:26:36.954 WARNING:teuthology.provision.fog.smithi012:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-11-14T06:26:38.988 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:26:42.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:26:42.046 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-14T06:26:42.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:26:43.299 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-14T06:26:43.300 DEBUG:teuthology.orchestra.run.smithi012:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-14T06:26:43.575 INFO:teuthology.provision.fog.smithi012:Node is ready 2023-11-14T06:26:43.576 DEBUG:teuthology.orchestra.run.smithi012:> hostname 2023-11-14T06:26:43.638 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012.front.sepia.ceph.com 2023-11-14T06:26:43.639 DEBUG:teuthology.orchestra.run.smithi012:> grep smithi012.front.sepia.ceph.com /etc/hosts 2023-11-14T06:26:43.700 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12 smithi012.front.sepia.ceph.com smithi012 2023-11-14T06:26:43.701 DEBUG:teuthology.orchestra.run.smithi012:> sudo hostname smithi012 2023-11-14T06:26:43.815 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i -e 's/smithi012.front.sepia.ceph.com/smithi012/g' /etc/hosts 2023-11-14T06:26:43.845 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i -e 's/172.21.15.12/172.21.15.12/g' /etc/hosts 2023-11-14T06:26:43.918 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i -e 's/smithi012.front.sepia.ceph.com/smithi012/g' /etc/hostname 2023-11-14T06:26:43.989 DEBUG:teuthology.orchestra.run.smithi012:> sudo hostname smithi012 2023-11-14T06:26:44.060 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2023-11-14T06:26:44.121 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="Red Hat Enterprise Linux" 2023-11-14T06:26:44.121 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="8.6 (Ootpa)" 2023-11-14T06:26:44.122 INFO:teuthology.orchestra.run.smithi012.stdout:ID="rhel" 2023-11-14T06:26:44.122 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE="fedora" 2023-11-14T06:26:44.122 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="8.6" 2023-11-14T06:26:44.122 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM_ID="platform:el8" 2023-11-14T06:26:44.122 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-14T06:26:44.123 INFO:teuthology.orchestra.run.smithi012.stdout:ANSI_COLOR="0;31" 2023-11-14T06:26:44.123 INFO:teuthology.orchestra.run.smithi012.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-14T06:26:44.123 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.redhat.com/" 2023-11-14T06:26:44.123 INFO:teuthology.orchestra.run.smithi012.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-14T06:26:44.123 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-14T06:26:44.124 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-14T06:26:44.124 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-14T06:26:44.124 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-14T06:26:44.124 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-14T06:26:44.124 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-14T06:26:44.125 INFO:teuthology.provision.fog.smithi012:Deploy complete! 2023-11-14T06:26:44.127 DEBUG:teuthology.parallel:result is None 2023-11-14T06:26:45.054 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-11-14T06:26:48.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:26:51.056 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:27:48.051 WARNING:teuthology.provision.fog.smithi148:timed out 2023-11-14T06:27:51.058 WARNING:teuthology.provision.fog.smithi029:timed out 2023-11-14T06:27:54.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:27:54.394 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-14T06:27:54.394 DEBUG:teuthology.orchestra.run.smithi148:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-14T06:27:54.705 INFO:teuthology.provision.fog.smithi148:Node is ready 2023-11-14T06:27:54.705 DEBUG:teuthology.orchestra.run.smithi148:> hostname 2023-11-14T06:27:54.723 INFO:teuthology.orchestra.run.smithi148.stdout:smithi148 2023-11-14T06:27:54.723 DEBUG:teuthology.orchestra.run.smithi148:> grep smithi148 /etc/hosts 2023-11-14T06:27:54.781 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.15.148 smithi148.front.sepia.ceph.com smithi148 2023-11-14T06:27:54.782 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname smithi148 2023-11-14T06:27:54.905 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i -e 's/smithi148/smithi148/g' /etc/hosts 2023-11-14T06:27:54.933 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i -e 's/172.21.15.148/172.21.15.148/g' /etc/hosts 2023-11-14T06:27:55.000 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i -e 's/smithi148/smithi148/g' /etc/hostname 2023-11-14T06:27:55.068 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname smithi148 2023-11-14T06:27:55.134 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2023-11-14T06:27:55.190 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="Red Hat Enterprise Linux" 2023-11-14T06:27:55.191 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="8.6 (Ootpa)" 2023-11-14T06:27:55.191 INFO:teuthology.orchestra.run.smithi148.stdout:ID="rhel" 2023-11-14T06:27:55.191 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="fedora" 2023-11-14T06:27:55.191 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="8.6" 2023-11-14T06:27:55.192 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el8" 2023-11-14T06:27:55.192 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-14T06:27:55.192 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2023-11-14T06:27:55.192 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-14T06:27:55.192 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://www.redhat.com/" 2023-11-14T06:27:55.193 INFO:teuthology.orchestra.run.smithi148.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-14T06:27:55.193 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-14T06:27:55.193 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-14T06:27:55.193 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-14T06:27:55.193 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-14T06:27:55.194 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-14T06:27:55.194 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-14T06:27:55.194 INFO:teuthology.provision.fog.smithi148:Deploy complete! 2023-11-14T06:27:55.195 DEBUG:teuthology.parallel:result is None 2023-11-14T06:27:57.060 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:27:57.394 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-14T06:27:57.394 DEBUG:teuthology.orchestra.run.smithi029:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-14T06:27:57.727 INFO:teuthology.provision.fog.smithi029:Node is ready 2023-11-14T06:27:57.727 DEBUG:teuthology.orchestra.run.smithi029:> hostname 2023-11-14T06:27:57.748 INFO:teuthology.orchestra.run.smithi029.stdout:smithi029.front.sepia.ceph.com 2023-11-14T06:27:57.749 DEBUG:teuthology.orchestra.run.smithi029:> grep smithi029.front.sepia.ceph.com /etc/hosts 2023-11-14T06:27:57.808 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29 smithi029.front.sepia.ceph.com smithi029 2023-11-14T06:27:57.809 DEBUG:teuthology.orchestra.run.smithi029:> sudo hostname smithi029 2023-11-14T06:27:57.943 DEBUG:teuthology.orchestra.run.smithi029:> sudo sed -i -e 's/smithi029.front.sepia.ceph.com/smithi029/g' /etc/hosts 2023-11-14T06:27:57.974 DEBUG:teuthology.orchestra.run.smithi029:> sudo sed -i -e 's/172.21.15.29/172.21.15.29/g' /etc/hosts 2023-11-14T06:27:58.043 DEBUG:teuthology.orchestra.run.smithi029:> sudo sed -i -e 's/smithi029.front.sepia.ceph.com/smithi029/g' /etc/hostname 2023-11-14T06:27:58.114 DEBUG:teuthology.orchestra.run.smithi029:> sudo hostname smithi029 2023-11-14T06:27:58.183 DEBUG:teuthology.orchestra.run.smithi029:> cat /etc/os-release 2023-11-14T06:27:58.241 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="Red Hat Enterprise Linux" 2023-11-14T06:27:58.242 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="8.6 (Ootpa)" 2023-11-14T06:27:58.242 INFO:teuthology.orchestra.run.smithi029.stdout:ID="rhel" 2023-11-14T06:27:58.242 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="fedora" 2023-11-14T06:27:58.242 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="8.6" 2023-11-14T06:27:58.242 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el8" 2023-11-14T06:27:58.243 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-14T06:27:58.243 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31" 2023-11-14T06:27:58.243 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-14T06:27:58.243 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://www.redhat.com/" 2023-11-14T06:27:58.243 INFO:teuthology.orchestra.run.smithi029.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-14T06:27:58.244 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-14T06:27:58.244 INFO:teuthology.orchestra.run.smithi029.stdout: 2023-11-14T06:27:58.244 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-14T06:27:58.244 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-14T06:27:58.244 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-14T06:27:58.245 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-14T06:27:58.245 INFO:teuthology.provision.fog.smithi029:Deploy complete! 2023-11-14T06:27:58.245 DEBUG:teuthology.parallel:result is None 2023-11-14T06:27:58.445 INFO:teuthology.lock.ops:Checking smithi012.front.sepia.ceph.com 2023-11-14T06:27:58.445 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-14T06:27:58.473 INFO:teuthology.lock.ops:Checking smithi029.front.sepia.ceph.com 2023-11-14T06:27:58.473 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-14T06:27:58.498 INFO:teuthology.lock.ops:Checking smithi148.front.sepia.ceph.com 2023-11-14T06:27:58.498 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-14T06:27:58.526 INFO:teuthology.lock.ops:Updating [smithi012.front.sepia.ceph.com]: set os type and version on server 2023-11-14T06:27:58.527 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-11-14T06:27:58.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:27:58.598 DEBUG:teuthology.orchestra.run.smithi012:> true 2023-11-14T06:27:58.733 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-11-14T06:27:58.733 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2023-11-14T06:27:58.791 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2023-11-14T06:27:58.792 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2023-11-14T06:27:58.851 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="Red Hat Enterprise Linux" 2023-11-14T06:27:58.852 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="8.6 (Ootpa)" 2023-11-14T06:27:58.852 INFO:teuthology.orchestra.run.smithi012.stdout:ID="rhel" 2023-11-14T06:27:58.852 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE="fedora" 2023-11-14T06:27:58.852 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="8.6" 2023-11-14T06:27:58.852 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM_ID="platform:el8" 2023-11-14T06:27:58.853 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-14T06:27:58.853 INFO:teuthology.orchestra.run.smithi012.stdout:ANSI_COLOR="0;31" 2023-11-14T06:27:58.853 INFO:teuthology.orchestra.run.smithi012.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-14T06:27:58.853 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.redhat.com/" 2023-11-14T06:27:58.853 INFO:teuthology.orchestra.run.smithi012.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-14T06:27:58.854 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-14T06:27:58.854 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-14T06:27:58.854 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-14T06:27:58.854 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-14T06:27:58.855 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-14T06:27:58.855 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-14T06:27:58.855 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2023-11-14T06:27:58.890 INFO:teuthology.lock.ops:Updating [smithi029.front.sepia.ceph.com]: set os type and version on server 2023-11-14T06:27:58.891 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2023-11-14T06:27:58.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:27:58.958 DEBUG:teuthology.orchestra.run.smithi029:> true 2023-11-14T06:27:59.034 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi029.front.sepia.ceph.com' 2023-11-14T06:27:59.035 DEBUG:teuthology.orchestra.run.smithi029:> uname -m 2023-11-14T06:27:59.094 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64 2023-11-14T06:27:59.095 DEBUG:teuthology.orchestra.run.smithi029:> cat /etc/os-release 2023-11-14T06:27:59.156 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="Red Hat Enterprise Linux" 2023-11-14T06:27:59.156 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="8.6 (Ootpa)" 2023-11-14T06:27:59.157 INFO:teuthology.orchestra.run.smithi029.stdout:ID="rhel" 2023-11-14T06:27:59.157 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="fedora" 2023-11-14T06:27:59.157 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="8.6" 2023-11-14T06:27:59.157 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el8" 2023-11-14T06:27:59.157 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-14T06:27:59.158 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31" 2023-11-14T06:27:59.158 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-14T06:27:59.158 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://www.redhat.com/" 2023-11-14T06:27:59.158 INFO:teuthology.orchestra.run.smithi029.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-14T06:27:59.158 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-14T06:27:59.159 INFO:teuthology.orchestra.run.smithi029.stdout: 2023-11-14T06:27:59.159 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-14T06:27:59.159 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-14T06:27:59.159 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-14T06:27:59.159 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-14T06:27:59.160 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2023-11-14T06:27:59.181 INFO:teuthology.lock.ops:Updating [smithi148.front.sepia.ceph.com]: set os type and version on server 2023-11-14T06:27:59.182 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-11-14T06:27:59.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:27:59.249 DEBUG:teuthology.orchestra.run.smithi148:> true 2023-11-14T06:27:59.318 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-11-14T06:27:59.318 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2023-11-14T06:27:59.376 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2023-11-14T06:27:59.377 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2023-11-14T06:27:59.434 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="Red Hat Enterprise Linux" 2023-11-14T06:27:59.434 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="8.6 (Ootpa)" 2023-11-14T06:27:59.435 INFO:teuthology.orchestra.run.smithi148.stdout:ID="rhel" 2023-11-14T06:27:59.435 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="fedora" 2023-11-14T06:27:59.435 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="8.6" 2023-11-14T06:27:59.435 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el8" 2023-11-14T06:27:59.435 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-14T06:27:59.436 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2023-11-14T06:27:59.436 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-14T06:27:59.436 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://www.redhat.com/" 2023-11-14T06:27:59.436 INFO:teuthology.orchestra.run.smithi148.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-14T06:27:59.436 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-14T06:27:59.437 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-11-14T06:27:59.437 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-14T06:27:59.437 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-14T06:27:59.437 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-14T06:27:59.437 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-14T06:27:59.438 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2023-11-14T06:27:59.456 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:27:59.562 INFO:teuthology.dispatcher.supervisor:Running job 7456878 2023-11-14T06:27:59.562 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-11-14_06:10:15-fs-wip-vshankar-testing-20231107.042705-distro-default-smithi/7456878 --name vshankar-2023-11-14_06:10:15-fs-wip-vshankar-testing-20231107.042705-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/misc}} -- /home/teuthworker/archive/vshankar-2023-11-14_06:10:15-fs-wip-vshankar-testing-20231107.042705-distro-default-smithi/7456878/orig.config.yaml 2023-11-14T06:27:59.566 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2023-11-14_06:10:15-fs-wip-vshankar-testing-20231107.042705-distro-default-smithi/7456878 2023-11-14T06:27:59.568 INFO:teuthology.dispatcher.supervisor:Job PID: 2545635 2023-11-14T06:27:59.568 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-14T06:29:59.569 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:31:59.654 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:33:59.718 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:35:59.784 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:37:59.850 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:39:59.915 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:41:59.980 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:44:00.047 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:46:00.109 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:48:00.175 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:50:00.229 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:52:00.288 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:54:00.360 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:56:00.427 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:58:00.492 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:00:00.556 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:02:00.630 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:04:00.693 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:06:00.766 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:08:00.839 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:10:00.913 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:12:00.984 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:14:01.065 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:16:01.136 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:18:01.200 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:20:01.273 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:22:01.329 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:24:01.399 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:26:01.465 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:28:01.538 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:30:01.618 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:32:01.690 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:34:01.754 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:36:01.826 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:38:01.890 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:40:01.962 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:42:02.026 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:44:02.097 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:46:02.170 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:48:02.253 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:50:02.321 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:52:02.394 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:54:02.457 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:56:02.530 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:58:02.594 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:00:02.667 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:02:02.739 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:04:02.818 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:06:02.890 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:08:02.953 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:10:03.016 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:12:03.082 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:14:03.154 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:16:03.238 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:18:03.310 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:20:03.372 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:22:03.436 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:24:03.500 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:26:03.591 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:28:03.664 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:30:03.727 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:32:03.792 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:34:03.855 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:36:03.922 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:38:03.977 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:40:04.041 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T08:40:04.099 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-11-14T08:40:04.305 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-14T08:40:04.306 INFO:teuthology.nuke:Checking targets against current locks 2023-11-14T08:40:04.354 DEBUG:teuthology.nuke:shortname: smithi012 2023-11-14T08:40:04.354 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-14T08:40:04.374 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-14_06:10:15-fs-wip-vshankar-testing-20231107.042705-distro-default-smithi/7456878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-14 06:22:30.990403', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-14T08:40:04.377 DEBUG:teuthology.nuke:shortname: smithi029 2023-11-14T08:40:04.378 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-14T08:40:04.395 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi029.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-14_06:10:15-fs-wip-vshankar-testing-20231107.042705-distro-default-smithi/7456878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-14 06:22:30.991456', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-14T08:40:04.399 DEBUG:teuthology.nuke:shortname: smithi148 2023-11-14T08:40:04.399 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-14T08:40:04.428 INFO:teuthology.orchestra.console.smithi012:Power off 2023-11-14T08:40:04.428 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-14T08:40:04.438 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-14_06:10:15-fs-wip-vshankar-testing-20231107.042705-distro-default-smithi/7456878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-14 06:22:30.992322', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-14T08:40:04.447 INFO:teuthology.orchestra.console.smithi029:Power off 2023-11-14T08:40:04.448 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-14T08:40:04.457 DEBUG:teuthology.orchestra.console.smithi012:power off output: Chassis Power Control: Down/Off 2023-11-14T08:40:04.458 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T08:40:04.469 DEBUG:teuthology.orchestra.console.smithi029:power off output: Chassis Power Control: Down/Off 2023-11-14T08:40:04.469 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T08:40:04.481 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is off 2023-11-14T08:40:04.490 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is off 2023-11-14T08:40:04.506 INFO:teuthology.orchestra.console.smithi148:Power off 2023-11-14T08:40:04.506 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-14T08:40:04.527 DEBUG:teuthology.orchestra.console.smithi148:power off output: Chassis Power Control: Down/Off 2023-11-14T08:40:04.527 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T08:40:04.547 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2023-11-14T08:40:04.582 INFO:teuthology.orchestra.console.smithi012:Power off completed 2023-11-14T08:40:04.590 INFO:teuthology.orchestra.console.smithi029:Power off completed 2023-11-14T08:40:04.648 INFO:teuthology.orchestra.console.smithi148:Power off completed 2023-11-14T08:40:04.732 INFO:teuthology.lock.ops:unlocked: smithi012.front.sepia.ceph.com 2023-11-14T08:40:04.740 INFO:teuthology.lock.ops:unlocked: smithi029.front.sepia.ceph.com 2023-11-14T08:40:04.800 INFO:teuthology.lock.ops:unlocked: smithi148.front.sepia.ceph.com