2023-11-04T02:34:09.321 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-04T02:34:09.323 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:34:09.495 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6656ddfc10>, signals=[15]) 2023-11-04T02:34:09.497 INFO:teuthology.lock.ops:Start node 'smithi032.front.sepia.ceph.com' reimaging 2023-11-04T02:34:09.497 INFO:teuthology.lock.ops:Updating [smithi032.front.sepia.ceph.com]: reset os type and version on server 2023-11-04T02:34:09.497 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2023-11-04T02:34:09.529 INFO:teuthology.lock.ops:Start node 'smithi086.front.sepia.ceph.com' reimaging 2023-11-04T02:34:09.529 INFO:teuthology.lock.ops:Updating [smithi086.front.sepia.ceph.com]: reset os type and version on server 2023-11-04T02:34:09.529 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2023-11-04T02:34:09.531 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /host {"name": "smithi032"} 2023-11-04T02:34:09.553 INFO:teuthology.lock.ops:Start node 'smithi097.front.sepia.ceph.com' reimaging 2023-11-04T02:34:09.554 INFO:teuthology.lock.ops:Updating [smithi097.front.sepia.ceph.com]: reset os type and version on server 2023-11-04T02:34:09.554 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2023-11-04T02:34:09.555 DEBUG:teuthology.provision.fog.smithi086:FOG request: GET /host {"name": "smithi086"} 2023-11-04T02:34:09.578 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /host {"name": "smithi097"} 2023-11-04T02:34:09.615 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-11-04T02:34:09.621 DEBUG:teuthology.provision.fog.smithi086:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-11-04T02:34:09.646 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-11-04T02:34:09.663 DEBUG:teuthology.provision.fog.smithi032:Requesting image smithi_rhel_8.6 (ID 5) 2023-11-04T02:34:09.663 DEBUG:teuthology.provision.fog.smithi032:FOG request: PUT /host/137 {"imageID": 5} 2023-11-04T02:34:09.666 DEBUG:teuthology.provision.fog.smithi086:Requesting image smithi_rhel_8.6 (ID 5) 2023-11-04T02:34:09.667 DEBUG:teuthology.provision.fog.smithi086:FOG request: PUT /host/191 {"imageID": 5} 2023-11-04T02:34:09.686 DEBUG:teuthology.provision.fog.smithi097:Requesting image smithi_rhel_8.6 (ID 5) 2023-11-04T02:34:09.686 DEBUG:teuthology.provision.fog.smithi097:FOG request: PUT /host/202 {"imageID": 5} 2023-11-04T02:34:09.713 INFO:teuthology.provision.fog.smithi086:Scheduling deploy of rhel 8.6 2023-11-04T02:34:09.713 DEBUG:teuthology.provision.fog.smithi086:FOG request: GET /task/active None 2023-11-04T02:34:09.716 INFO:teuthology.provision.fog.smithi032:Scheduling deploy of rhel 8.6 2023-11-04T02:34:09.717 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-11-04T02:34:09.735 INFO:teuthology.provision.fog.smithi097:Scheduling deploy of rhel 8.6 2023-11-04T02:34:09.735 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-11-04T02:34:09.781 DEBUG:teuthology.provision.fog.smithi086:FOG request: GET /tasktype {"name": "deploy"} 2023-11-04T02:34:09.785 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /tasktype {"name": "deploy"} 2023-11-04T02:34:09.792 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /tasktype {"name": "deploy"} 2023-11-04T02:34:09.825 DEBUG:teuthology.provision.fog.smithi097:FOG request: POST /host/202/task {"taskTypeID": 1} 2023-11-04T02:34:09.829 DEBUG:teuthology.provision.fog.smithi032:FOG request: POST /host/137/task {"taskTypeID": 1} 2023-11-04T02:34:09.832 DEBUG:teuthology.provision.fog.smithi086:FOG request: POST /host/191/task {"taskTypeID": 1} 2023-11-04T02:34:09.880 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-11-04T02:34:09.997 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-11-04T02:34:10.007 DEBUG:teuthology.provision.fog.smithi086:FOG request: GET /task/active None 2023-11-04T02:34:10.086 INFO:teuthology.orchestra.console.smithi032:Power off 2023-11-04T02:34:10.087 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-04T02:34:10.100 INFO:teuthology.orchestra.console.smithi097:Power off 2023-11-04T02:34:10.101 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-04T02:34:10.109 DEBUG:teuthology.orchestra.console.smithi032:power off output: Chassis Power Control: Down/Off 2023-11-04T02:34:10.110 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T02:34:10.123 INFO:teuthology.orchestra.console.smithi086:Power off 2023-11-04T02:34:10.123 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-04T02:34:10.132 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2023-11-04T02:34:10.132 DEBUG:teuthology.orchestra.console.smithi097:power off output: Chassis Power Control: Down/Off 2023-11-04T02:34:10.133 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T02:34:10.144 DEBUG:teuthology.orchestra.console.smithi086:power off output: Chassis Power Control: Down/Off 2023-11-04T02:34:10.145 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T02:34:10.155 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2023-11-04T02:34:10.169 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2023-11-04T02:34:10.233 INFO:teuthology.orchestra.console.smithi032:Power off completed 2023-11-04T02:34:10.257 INFO:teuthology.orchestra.console.smithi097:Power off completed 2023-11-04T02:34:10.272 INFO:teuthology.orchestra.console.smithi086:Power off completed 2023-11-04T02:34:10.334 INFO:teuthology.orchestra.console.smithi032:Power on 2023-11-04T02:34:10.335 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-04T02:34:10.358 INFO:teuthology.orchestra.console.smithi097:Power on 2023-11-04T02:34:10.359 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-04T02:34:10.367 DEBUG:teuthology.orchestra.console.smithi032:power on output: Chassis Power Control: Up/On 2023-11-04T02:34:10.368 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T02:34:10.375 INFO:teuthology.orchestra.console.smithi086:Power on 2023-11-04T02:34:10.376 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-04T02:34:10.385 DEBUG:teuthology.orchestra.console.smithi097:power on output: Chassis Power Control: Up/On 2023-11-04T02:34:10.386 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T02:34:10.395 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2023-11-04T02:34:10.398 DEBUG:teuthology.orchestra.console.smithi086:power on output: Chassis Power Control: Up/On 2023-11-04T02:34:10.398 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T02:34:10.407 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2023-11-04T02:34:10.420 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2023-11-04T02:34:14.399 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T02:34:14.409 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T02:34:14.421 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T02:34:14.509 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2023-11-04T02:34:14.518 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2023-11-04T02:34:14.531 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2023-11-04T02:34:18.514 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T02:34:18.528 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T02:34:18.537 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T02:34:18.630 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2023-11-04T02:34:18.638 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2023-11-04T02:34:18.732 INFO:teuthology.orchestra.console.smithi032:Power on completed 2023-11-04T02:34:18.737 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2023-11-04T02:34:18.740 INFO:teuthology.orchestra.console.smithi097:Power on completed 2023-11-04T02:34:18.833 INFO:teuthology.provision.fog.smithi032:Waiting for deploy to finish 2023-11-04T02:34:18.833 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-11-04T02:34:18.838 INFO:teuthology.orchestra.console.smithi086:Power on completed 2023-11-04T02:34:18.842 INFO:teuthology.provision.fog.smithi097:Waiting for deploy to finish 2023-11-04T02:34:18.842 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-11-04T02:34:18.939 INFO:teuthology.provision.fog.smithi086:Waiting for deploy to finish 2023-11-04T02:34:18.939 DEBUG:teuthology.provision.fog.smithi086:FOG request: GET /task/active None 2023-11-04T02:34:33.936 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-11-04T02:34:33.937 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-11-04T02:34:34.017 DEBUG:teuthology.provision.fog.smithi086:FOG request: GET /task/active None 2023-11-04T02:34:49.051 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-11-04T02:34:49.054 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-11-04T02:34:49.105 DEBUG:teuthology.provision.fog.smithi086:FOG request: GET /task/active None 2023-11-04T02:35:04.170 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-11-04T02:35:04.172 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-11-04T02:35:04.187 DEBUG:teuthology.provision.fog.smithi086:FOG request: GET /task/active None 2023-11-04T02:35:19.282 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-11-04T02:35:19.283 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-11-04T02:35:19.289 DEBUG:teuthology.provision.fog.smithi086:FOG request: GET /task/active None 2023-11-04T02:35:34.390 DEBUG:teuthology.provision.fog.smithi086:FOG request: GET /task/active None 2023-11-04T02:35:34.392 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-11-04T02:35:34.393 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-11-04T02:35:49.506 DEBUG:teuthology.provision.fog.smithi086:FOG request: GET /task/active None 2023-11-04T02:35:49.509 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-11-04T02:35:49.511 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-11-04T02:36:04.626 DEBUG:teuthology.provision.fog.smithi086:FOG request: GET /task/active None 2023-11-04T02:36:04.628 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-11-04T02:36:04.630 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-11-04T02:36:19.734 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-11-04T02:36:19.736 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-11-04T02:36:19.737 DEBUG:teuthology.provision.fog.smithi086:FOG request: GET /task/active None 2023-11-04T02:36:34.839 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-11-04T02:36:34.841 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-11-04T02:36:34.842 DEBUG:teuthology.provision.fog.smithi086:FOG request: GET /task/active None 2023-11-04T02:36:49.944 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-11-04T02:36:49.946 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-11-04T02:36:49.947 DEBUG:teuthology.provision.fog.smithi086:FOG request: GET /task/active None 2023-11-04T02:36:50.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:36:53.114 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-11-04T02:36:59.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:37:02.174 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-11-04T02:37:05.026 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-11-04T02:37:05.037 DEBUG:teuthology.provision.fog.smithi086:FOG request: GET /task/active None 2023-11-04T02:37:05.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:37:05.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:37:08.158 WARNING:teuthology.provision.fog.smithi097:[Errno None] Unable to connect to port 22 on 172.21.15.97 2023-11-04T02:37:08.176 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:37:08.190 WARNING:teuthology.provision.fog.smithi086:[Errno None] Unable to connect to port 22 on 172.21.15.86 2023-11-04T02:37:11.226 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-11-04T02:37:14.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:37:14.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:37:17.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:37:17.246 WARNING:teuthology.provision.fog.smithi097:[Errno None] Unable to connect to port 22 on 172.21.15.97 2023-11-04T02:37:17.247 WARNING:teuthology.provision.fog.smithi086:[Errno None] Unable to connect to port 22 on 172.21.15.86 2023-11-04T02:37:20.282 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-11-04T02:37:23.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:37:23.253 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:37:26.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:37:26.330 WARNING:teuthology.provision.fog.smithi097:[Errno None] Unable to connect to port 22 on 172.21.15.97 2023-11-04T02:37:26.331 WARNING:teuthology.provision.fog.smithi086:[Errno None] Unable to connect to port 22 on 172.21.15.86 2023-11-04T02:37:32.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:37:32.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:37:35.386 WARNING:teuthology.provision.fog.smithi097:[Errno None] Unable to connect to port 22 on 172.21.15.97 2023-11-04T02:37:35.418 WARNING:teuthology.provision.fog.smithi086:[Errno None] Unable to connect to port 22 on 172.21.15.86 2023-11-04T02:37:41.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:37:41.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:37:44.442 WARNING:teuthology.provision.fog.smithi097:[Errno None] Unable to connect to port 22 on 172.21.15.97 2023-11-04T02:37:44.474 WARNING:teuthology.provision.fog.smithi086:[Errno None] Unable to connect to port 22 on 172.21.15.86 2023-11-04T02:37:50.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:37:50.476 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:37:53.502 WARNING:teuthology.provision.fog.smithi097:[Errno None] Unable to connect to port 22 on 172.21.15.97 2023-11-04T02:37:53.530 WARNING:teuthology.provision.fog.smithi086:[Errno None] Unable to connect to port 22 on 172.21.15.86 2023-11-04T02:37:59.509 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:37:59.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:38:26.314 WARNING:teuthology.provision.fog.smithi032:timed out 2023-11-04T02:38:32.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:38:32.552 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-04T02:38:32.553 DEBUG:teuthology.orchestra.run.smithi032:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-04T02:38:32.834 INFO:teuthology.provision.fog.smithi032:Node is ready 2023-11-04T02:38:32.834 DEBUG:teuthology.orchestra.run.smithi032:> hostname 2023-11-04T02:38:32.853 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032.front.sepia.ceph.com 2023-11-04T02:38:32.854 DEBUG:teuthology.orchestra.run.smithi032:> grep smithi032.front.sepia.ceph.com /etc/hosts 2023-11-04T02:38:32.913 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32 smithi032.front.sepia.ceph.com smithi032 2023-11-04T02:38:32.913 DEBUG:teuthology.orchestra.run.smithi032:> sudo hostname smithi032 2023-11-04T02:38:33.140 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i -e 's/smithi032.front.sepia.ceph.com/smithi032/g' /etc/hosts 2023-11-04T02:38:33.171 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i -e 's/172.21.15.32/172.21.15.32/g' /etc/hosts 2023-11-04T02:38:33.241 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i -e 's/smithi032.front.sepia.ceph.com/smithi032/g' /etc/hostname 2023-11-04T02:38:33.311 DEBUG:teuthology.orchestra.run.smithi032:> sudo hostname smithi032 2023-11-04T02:38:33.379 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2023-11-04T02:38:33.437 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Red Hat Enterprise Linux" 2023-11-04T02:38:33.438 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="8.6 (Ootpa)" 2023-11-04T02:38:33.438 INFO:teuthology.orchestra.run.smithi032.stdout:ID="rhel" 2023-11-04T02:38:33.438 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="fedora" 2023-11-04T02:38:33.438 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="8.6" 2023-11-04T02:38:33.439 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el8" 2023-11-04T02:38:33.439 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-04T02:38:33.439 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2023-11-04T02:38:33.439 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-04T02:38:33.439 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.redhat.com/" 2023-11-04T02:38:33.439 INFO:teuthology.orchestra.run.smithi032.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-04T02:38:33.440 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-04T02:38:33.440 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-04T02:38:33.440 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-04T02:38:33.440 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-04T02:38:33.440 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-04T02:38:33.441 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-04T02:38:33.441 INFO:teuthology.provision.fog.smithi032:Deploy complete! 2023-11-04T02:38:33.442 DEBUG:teuthology.parallel:result is None 2023-11-04T02:38:59.534 WARNING:teuthology.provision.fog.smithi097:timed out 2023-11-04T02:38:59.535 WARNING:teuthology.provision.fog.smithi086:timed out 2023-11-04T02:39:05.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:39:05.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:39:05.870 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-04T02:39:05.870 DEBUG:teuthology.orchestra.run.smithi097:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-04T02:39:05.898 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-04T02:39:05.898 DEBUG:teuthology.orchestra.run.smithi086:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-04T02:39:06.161 INFO:teuthology.provision.fog.smithi097:Node is ready 2023-11-04T02:39:06.162 DEBUG:teuthology.orchestra.run.smithi097:> hostname 2023-11-04T02:39:06.181 INFO:teuthology.orchestra.run.smithi097.stdout:smithi097 2023-11-04T02:39:06.182 DEBUG:teuthology.orchestra.run.smithi097:> grep smithi097 /etc/hosts 2023-11-04T02:39:06.207 INFO:teuthology.provision.fog.smithi086:Node is ready 2023-11-04T02:39:06.208 DEBUG:teuthology.orchestra.run.smithi086:> hostname 2023-11-04T02:39:06.225 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086 2023-11-04T02:39:06.226 DEBUG:teuthology.orchestra.run.smithi086:> grep smithi086 /etc/hosts 2023-11-04T02:39:06.239 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97 smithi097.front.sepia.ceph.com smithi097 2023-11-04T02:39:06.240 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname smithi097 2023-11-04T02:39:06.286 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86 smithi086.front.sepia.ceph.com smithi086 2023-11-04T02:39:06.287 DEBUG:teuthology.orchestra.run.smithi086:> sudo hostname smithi086 2023-11-04T02:39:06.350 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i -e 's/smithi097/smithi097/g' /etc/hosts 2023-11-04T02:39:06.379 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i -e 's/172.21.15.97/172.21.15.97/g' /etc/hosts 2023-11-04T02:39:06.402 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i -e 's/smithi086/smithi086/g' /etc/hosts 2023-11-04T02:39:06.430 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i -e 's/172.21.15.86/172.21.15.86/g' /etc/hosts 2023-11-04T02:39:06.446 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i -e 's/smithi097/smithi097/g' /etc/hostname 2023-11-04T02:39:06.501 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i -e 's/smithi086/smithi086/g' /etc/hostname 2023-11-04T02:39:06.513 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname smithi097 2023-11-04T02:39:06.571 DEBUG:teuthology.orchestra.run.smithi086:> sudo hostname smithi086 2023-11-04T02:39:06.579 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2023-11-04T02:39:06.636 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="Red Hat Enterprise Linux" 2023-11-04T02:39:06.636 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="8.6 (Ootpa)" 2023-11-04T02:39:06.636 INFO:teuthology.orchestra.run.smithi097.stdout:ID="rhel" 2023-11-04T02:39:06.637 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="fedora" 2023-11-04T02:39:06.637 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="8.6" 2023-11-04T02:39:06.637 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el8" 2023-11-04T02:39:06.637 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-04T02:39:06.637 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2023-11-04T02:39:06.637 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-04T02:39:06.638 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://www.redhat.com/" 2023-11-04T02:39:06.638 INFO:teuthology.orchestra.run.smithi097.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-04T02:39:06.638 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-04T02:39:06.638 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-04T02:39:06.639 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-04T02:39:06.639 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-04T02:39:06.639 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-04T02:39:06.640 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-04T02:39:06.640 INFO:teuthology.provision.fog.smithi097:Deploy complete! 2023-11-04T02:39:06.641 DEBUG:teuthology.parallel:result is None 2023-11-04T02:39:06.642 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2023-11-04T02:39:06.702 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="Red Hat Enterprise Linux" 2023-11-04T02:39:06.702 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="8.6 (Ootpa)" 2023-11-04T02:39:06.703 INFO:teuthology.orchestra.run.smithi086.stdout:ID="rhel" 2023-11-04T02:39:06.703 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="fedora" 2023-11-04T02:39:06.703 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="8.6" 2023-11-04T02:39:06.703 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el8" 2023-11-04T02:39:06.704 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-04T02:39:06.704 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2023-11-04T02:39:06.704 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-04T02:39:06.704 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://www.redhat.com/" 2023-11-04T02:39:06.704 INFO:teuthology.orchestra.run.smithi086.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-04T02:39:06.704 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-04T02:39:06.705 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-11-04T02:39:06.705 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-04T02:39:06.705 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-04T02:39:06.705 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-04T02:39:06.706 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-04T02:39:06.706 INFO:teuthology.provision.fog.smithi086:Deploy complete! 2023-11-04T02:39:06.707 DEBUG:teuthology.parallel:result is None 2023-11-04T02:39:06.900 INFO:teuthology.lock.ops:Checking smithi032.front.sepia.ceph.com 2023-11-04T02:39:06.901 INFO:teuthology.lock.ops:Checking smithi086.front.sepia.ceph.com 2023-11-04T02:39:06.901 INFO:teuthology.lock.ops:Checking smithi097.front.sepia.ceph.com 2023-11-04T02:39:06.901 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-04T02:39:06.932 INFO:teuthology.lock.ops:Updating [smithi032.front.sepia.ceph.com]: set os type and version on server 2023-11-04T02:39:06.933 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-11-04T02:39:06.934 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:39:07.003 DEBUG:teuthology.orchestra.run.smithi032:> true 2023-11-04T02:39:07.137 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-11-04T02:39:07.137 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2023-11-04T02:39:07.195 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2023-11-04T02:39:07.196 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2023-11-04T02:39:07.254 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Red Hat Enterprise Linux" 2023-11-04T02:39:07.255 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="8.6 (Ootpa)" 2023-11-04T02:39:07.255 INFO:teuthology.orchestra.run.smithi032.stdout:ID="rhel" 2023-11-04T02:39:07.255 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="fedora" 2023-11-04T02:39:07.255 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="8.6" 2023-11-04T02:39:07.256 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el8" 2023-11-04T02:39:07.256 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-04T02:39:07.256 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2023-11-04T02:39:07.256 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-04T02:39:07.256 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.redhat.com/" 2023-11-04T02:39:07.257 INFO:teuthology.orchestra.run.smithi032.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-04T02:39:07.257 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-04T02:39:07.257 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-11-04T02:39:07.257 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-04T02:39:07.257 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-04T02:39:07.258 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-04T02:39:07.258 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-04T02:39:07.259 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2023-11-04T02:39:07.281 INFO:teuthology.lock.ops:Updating [smithi086.front.sepia.ceph.com]: set os type and version on server 2023-11-04T02:39:07.281 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2023-11-04T02:39:07.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:39:07.352 DEBUG:teuthology.orchestra.run.smithi086:> true 2023-11-04T02:39:07.420 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2023-11-04T02:39:07.420 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2023-11-04T02:39:07.481 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2023-11-04T02:39:07.482 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2023-11-04T02:39:07.542 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="Red Hat Enterprise Linux" 2023-11-04T02:39:07.543 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="8.6 (Ootpa)" 2023-11-04T02:39:07.543 INFO:teuthology.orchestra.run.smithi086.stdout:ID="rhel" 2023-11-04T02:39:07.543 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="fedora" 2023-11-04T02:39:07.543 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="8.6" 2023-11-04T02:39:07.543 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el8" 2023-11-04T02:39:07.544 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-04T02:39:07.544 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2023-11-04T02:39:07.544 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-04T02:39:07.544 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://www.redhat.com/" 2023-11-04T02:39:07.544 INFO:teuthology.orchestra.run.smithi086.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-04T02:39:07.545 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-04T02:39:07.545 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-11-04T02:39:07.545 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-04T02:39:07.545 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-04T02:39:07.546 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-04T02:39:07.546 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-04T02:39:07.546 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2023-11-04T02:39:07.573 INFO:teuthology.lock.ops:Updating [smithi097.front.sepia.ceph.com]: set os type and version on server 2023-11-04T02:39:07.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-11-04T02:39:07.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:39:07.641 DEBUG:teuthology.orchestra.run.smithi097:> true 2023-11-04T02:39:07.709 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-11-04T02:39:07.709 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2023-11-04T02:39:07.769 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2023-11-04T02:39:07.770 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2023-11-04T02:39:07.829 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="Red Hat Enterprise Linux" 2023-11-04T02:39:07.829 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="8.6 (Ootpa)" 2023-11-04T02:39:07.829 INFO:teuthology.orchestra.run.smithi097.stdout:ID="rhel" 2023-11-04T02:39:07.829 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="fedora" 2023-11-04T02:39:07.830 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="8.6" 2023-11-04T02:39:07.830 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el8" 2023-11-04T02:39:07.830 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-04T02:39:07.830 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2023-11-04T02:39:07.831 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-04T02:39:07.831 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://www.redhat.com/" 2023-11-04T02:39:07.831 INFO:teuthology.orchestra.run.smithi097.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-04T02:39:07.831 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-04T02:39:07.831 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-04T02:39:07.831 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-04T02:39:07.832 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-04T02:39:07.832 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-04T02:39:07.832 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-04T02:39:07.833 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2023-11-04T02:39:07.856 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:39:07.966 INFO:teuthology.dispatcher.supervisor:Running job 7445787 2023-11-04T02:39:07.967 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-11-03_16:49:22-fs-rishabh-unqiue-fsnames-testing-default-smithi/7445787 --name rishabh-2023-11-03_16:49:22-fs-rishabh-unqiue-fsnames-testing-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/fsx}} -- /home/teuthworker/archive/rishabh-2023-11-03_16:49:22-fs-rishabh-unqiue-fsnames-testing-default-smithi/7445787/orig.config.yaml 2023-11-04T02:39:07.970 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-11-03_16:49:22-fs-rishabh-unqiue-fsnames-testing-default-smithi/7445787 2023-11-04T02:39:07.971 INFO:teuthology.dispatcher.supervisor:Job PID: 264383 2023-11-04T02:39:07.971 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-04T02:41:08.070 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:43:08.243 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:45:08.407 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:47:08.567 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:49:08.732 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:51:08.901 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:53:09.071 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:55:09.243 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:57:09.415 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:59:09.587 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T03:01:09.772 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T03:03:09.943 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T03:05:10.107 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T03:07:10.271 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T03:09:10.439 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T03:11:10.615 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T03:13:10.791 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T03:15:10.959 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T03:17:11.132 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T03:17:11.190 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-11-04T03:17:11.406 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-04T03:17:11.406 INFO:teuthology.nuke:Checking targets against current locks 2023-11-04T03:17:11.454 DEBUG:teuthology.nuke:shortname: smithi032 2023-11-04T03:17:11.454 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-04T03:17:11.483 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-03_16:49:22-fs-rishabh-unqiue-fsnames-testing-default-smithi/7445787', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-04 02:34:08.485345', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-04T03:17:11.489 DEBUG:teuthology.nuke:shortname: smithi086 2023-11-04T03:17:11.490 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-04T03:17:11.514 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-03_16:49:22-fs-rishabh-unqiue-fsnames-testing-default-smithi/7445787', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-04 02:34:08.487105', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-04T03:17:11.515 DEBUG:teuthology.nuke:shortname: smithi097 2023-11-04T03:17:11.515 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-04T03:17:11.538 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-03_16:49:22-fs-rishabh-unqiue-fsnames-testing-default-smithi/7445787', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-04 02:34:08.483334', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-04T03:17:11.551 INFO:teuthology.orchestra.console.smithi032:Power off 2023-11-04T03:17:11.552 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-04T03:17:11.574 DEBUG:teuthology.orchestra.console.smithi032:power off output: Chassis Power Control: Down/Off 2023-11-04T03:17:11.574 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T03:17:11.588 INFO:teuthology.orchestra.console.smithi086:Power off 2023-11-04T03:17:11.589 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-04T03:17:11.602 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2023-11-04T03:17:11.611 DEBUG:teuthology.orchestra.console.smithi086:power off output: Chassis Power Control: Down/Off 2023-11-04T03:17:11.612 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T03:17:11.630 INFO:teuthology.orchestra.console.smithi097:Power off 2023-11-04T03:17:11.631 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-04T03:17:11.639 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2023-11-04T03:17:11.703 INFO:teuthology.orchestra.console.smithi032:Power off completed 2023-11-04T03:17:11.741 INFO:teuthology.orchestra.console.smithi086:Power off completed 2023-11-04T03:17:11.847 INFO:teuthology.lock.ops:unlocked: smithi032.front.sepia.ceph.com 2023-11-04T03:17:11.880 INFO:teuthology.lock.ops:unlocked: smithi086.front.sepia.ceph.com 2023-11-04T03:17:31.700 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-11-04T03:17:31.701 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-04T03:17:51.774 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-11-04T03:17:51.775 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-04T03:18:11.844 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-11-04T03:18:11.845 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T03:18:12.854 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-11-04T03:18:16.858 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T03:18:17.966 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-11-04T03:18:21.970 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T03:18:23.079 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-11-04T03:18:27.082 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T03:18:28.196 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-11-04T03:18:32.201 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T03:18:33.310 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-11-04T03:18:37.315 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T03:18:38.424 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-11-04T03:18:42.426 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T03:18:43.534 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-11-04T03:18:47.538 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T03:18:48.647 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-11-04T03:18:52.650 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T03:18:53.759 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-11-04T03:18:57.763 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T03:18:58.871 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-11-04T03:19:02.874 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T03:19:03.983 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-11-04T03:19:07.987 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T03:19:09.100 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-11-04T03:19:13.104 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T03:19:14.213 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-11-04T03:19:18.218 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T03:19:19.328 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-11-04T03:19:23.331 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T03:19:24.441 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-11-04T03:19:24.441 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds 2023-11-04T03:19:24.442 ERROR:teuthology.orchestra.console.smithi097:Failed to power off 2023-11-04T03:19:24.488 INFO:teuthology.lock.ops:unlocked: smithi097.front.sepia.ceph.com