2023-07-14T04:26:20.381 INFO:root:teuthology version: 0.0.1.dev143+g073e6eb 2023-07-14T04:26:20.383 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:26:20.383 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:26:20.589 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f64609368b0>, signals=[15]) 2023-07-14T04:26:20.590 INFO:teuthology.lock.ops:Start node 'smithi012.front.sepia.ceph.com' reimaging 2023-07-14T04:26:20.591 INFO:teuthology.lock.ops:Updating [smithi012.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T04:26:20.591 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2023-07-14T04:26:20.620 INFO:teuthology.lock.ops:Start node 'smithi125.front.sepia.ceph.com' reimaging 2023-07-14T04:26:20.620 INFO:teuthology.lock.ops:Updating [smithi125.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T04:26:20.620 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server 2023-07-14T04:26:20.622 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /host {"name": "smithi012"} 2023-07-14T04:26:24.078 INFO:teuthology.lock.ops:Start node 'smithi177.front.sepia.ceph.com' reimaging 2023-07-14T04:26:24.079 INFO:teuthology.lock.ops:Updating [smithi177.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T04:26:24.079 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2023-07-14T04:26:24.080 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /host {"name": "smithi125"} 2023-07-14T04:26:44.098 DEBUG:teuthology.provision.fog.smithi177:FOG request: GET /host {"name": "smithi177"} 2023-07-14T04:26:49.133 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-14T04:26:51.930 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-14T04:27:01.972 DEBUG:teuthology.provision.fog.smithi177:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-14T04:27:06.981 DEBUG:teuthology.provision.fog.smithi012:Requesting image smithi_centos_9.stream (ID 10) 2023-07-14T04:27:06.981 DEBUG:teuthology.provision.fog.smithi012:FOG request: PUT /host/118 {"imageID": 10} 2023-07-14T04:27:06.982 DEBUG:teuthology.provision.fog.smithi125:Requesting image smithi_centos_9.stream (ID 10) 2023-07-14T04:27:06.982 DEBUG:teuthology.provision.fog.smithi125:FOG request: PUT /host/230 {"imageID": 10} 2023-07-14T04:27:07.040 DEBUG:teuthology.provision.fog.smithi177:Requesting image smithi_centos_9.stream (ID 10) 2023-07-14T04:27:07.041 DEBUG:teuthology.provision.fog.smithi177:FOG request: PUT /host/282 {"imageID": 10} 2023-07-14T04:27:07.044 INFO:teuthology.provision.fog.smithi125:Scheduling deploy of centos 9.stream 2023-07-14T04:27:07.045 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-07-14T04:27:07.065 INFO:teuthology.provision.fog.smithi012:Scheduling deploy of centos 9.stream 2023-07-14T04:27:07.066 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /task/active None 2023-07-14T04:27:07.103 INFO:teuthology.provision.fog.smithi177:Scheduling deploy of centos 9.stream 2023-07-14T04:27:07.103 DEBUG:teuthology.provision.fog.smithi177:FOG request: GET /task/active None 2023-07-14T04:27:07.140 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T04:27:07.155 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T04:27:07.181 DEBUG:teuthology.provision.fog.smithi177:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T04:27:12.187 DEBUG:teuthology.provision.fog.smithi012:FOG request: POST /host/118/task {"taskTypeID": 1} 2023-07-14T04:27:12.188 DEBUG:teuthology.provision.fog.smithi125:FOG request: POST /host/230/task {"taskTypeID": 1} 2023-07-14T04:27:22.248 DEBUG:teuthology.provision.fog.smithi177:FOG request: POST /host/282/task {"taskTypeID": 1} 2023-07-14T04:27:22.260 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /task/active None 2023-07-14T04:27:22.263 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-07-14T04:27:22.362 DEBUG:teuthology.provision.fog.smithi177:FOG request: GET /task/active None 2023-07-14T04:27:32.461 INFO:teuthology.orchestra.console:Power off smithi125 2023-07-14T04:27:32.462 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T04:27:34.934 INFO:teuthology.orchestra.console:Power off smithi177 2023-07-14T04:27:34.935 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T04:27:34.952 INFO:teuthology.orchestra.console:Power off smithi012 2023-07-14T04:27:34.953 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T04:27:34.962 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:27:34.974 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:27:37.486 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:27:37.607 INFO:teuthology.orchestra.console:Power off for smithi125 completed 2023-07-14T04:27:37.708 INFO:teuthology.orchestra.console:Power on smithi125 2023-07-14T04:27:37.708 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T04:27:37.729 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:27:38.988 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:27:38.998 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:27:41.753 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:27:43.099 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:27:43.110 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:27:45.864 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:27:46.075 INFO:teuthology.orchestra.console:Power on for smithi125 completed 2023-07-14T04:27:46.176 INFO:teuthology.provision.fog.smithi125:Waiting for deploy to finish 2023-07-14T04:27:46.177 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-07-14T04:27:51.185 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:27:51.195 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:27:51.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:27:56.360 INFO:teuthology.orchestra.console:Power off for smithi177 completed 2023-07-14T04:27:56.360 INFO:teuthology.orchestra.console:Power off for smithi012 completed 2023-07-14T04:27:56.461 INFO:teuthology.orchestra.console:Power on smithi177 2023-07-14T04:27:56.461 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T04:27:56.470 INFO:teuthology.orchestra.console:Power on smithi012 2023-07-14T04:27:56.471 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T04:27:59.312 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-07-14T04:28:01.499 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:28:05.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:28:10.324 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:28:11.492 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:28:13.392 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-07-14T04:28:15.436 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:28:16.504 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:28:19.397 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:28:20.547 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:28:21.616 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:28:22.452 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-07-14T04:28:25.657 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:28:26.729 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:28:28.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:28:33.463 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:28:33.475 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:28:36.528 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-07-14T04:28:38.579 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:28:38.589 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:28:42.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:28:47.538 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:28:47.545 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:28:50.608 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-07-14T04:28:52.648 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:28:52.655 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:28:56.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:29:01.617 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:29:01.626 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:29:06.729 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:29:06.739 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:29:07.898 INFO:teuthology.orchestra.console:Power on for smithi177 completed 2023-07-14T04:29:07.999 INFO:teuthology.provision.fog.smithi177:Waiting for deploy to finish 2023-07-14T04:29:08.000 DEBUG:teuthology.provision.fog.smithi177:FOG request: GET /task/active None 2023-07-14T04:29:13.005 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:29:13.119 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:29:18.227 INFO:teuthology.orchestra.console:Power on for smithi012 completed 2023-07-14T04:29:18.327 INFO:teuthology.provision.fog.smithi012:Waiting for deploy to finish 2023-07-14T04:29:18.328 DEBUG:teuthology.provision.fog.smithi012:FOG request: GET /task/active None 2023-07-14T04:29:23.329 WARNING:teuthology.provision.fog.smithi177:[Errno None] Unable to connect to port 22 on 172.21.15.177 2023-07-14T04:29:23.330 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-07-14T04:29:23.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:29:29.332 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:29:29.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:29:29.336 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-07-14T04:29:31.476 WARNING:teuthology.provision.fog.smithi012:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-07-14T04:29:35.340 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:29:35.542 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-14T04:29:35.543 DEBUG:teuthology.orchestra.run.smithi125:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-14T04:29:36.048 INFO:teuthology.provision.fog.smithi125:Node is ready 2023-07-14T04:29:36.049 DEBUG:teuthology.orchestra.run.smithi125:> hostname 2023-07-14T04:29:36.055 INFO:teuthology.orchestra.run.smithi125.stdout:smithi125.front.sepia.ceph.com 2023-07-14T04:29:36.055 DEBUG:teuthology.orchestra.run.smithi125:> grep smithi125.front.sepia.ceph.com /etc/hosts 2023-07-14T04:29:36.104 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.125 smithi125.front.sepia.ceph.com smithi125 2023-07-14T04:29:36.105 DEBUG:teuthology.orchestra.run.smithi125:> sudo hostname smithi125 2023-07-14T04:29:36.208 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/smithi125.front.sepia.ceph.com/smithi125/g' /etc/hosts 2023-07-14T04:29:36.225 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/172.21.15.125/172.21.15.125/g' /etc/hosts 2023-07-14T04:29:36.282 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/smithi125.front.sepia.ceph.com/smithi125/g' /etc/hostname 2023-07-14T04:29:36.292 DEBUG:teuthology.orchestra.run.smithi125:> sudo hostname smithi125 2023-07-14T04:29:36.301 DEBUG:teuthology.orchestra.run.smithi125:> cat /etc/os-release 2023-07-14T04:29:36.307 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="Ubuntu" 2023-07-14T04:29:36.308 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-07-14T04:29:36.308 INFO:teuthology.orchestra.run.smithi125.stdout:ID=ubuntu 2023-07-14T04:29:36.308 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE=debian 2023-07-14T04:29:36.308 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-07-14T04:29:36.308 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="20.04" 2023-07-14T04:29:36.309 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://www.ubuntu.com/" 2023-07-14T04:29:36.309 INFO:teuthology.orchestra.run.smithi125.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-07-14T04:29:36.309 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-07-14T04:29:36.309 INFO:teuthology.orchestra.run.smithi125.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-07-14T04:29:36.309 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_CODENAME=focal 2023-07-14T04:29:36.309 INFO:teuthology.orchestra.run.smithi125.stdout:UBUNTU_CODENAME=focal 2023-07-14T04:29:36.312 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi125's OS to be centos 9 but found ubuntu 20.04 2023-07-14T04:29:36.313 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:29:36.313 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:29:41.322 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:29:46.409 INFO:teuthology.nuke:Checking targets against current locks 2023-07-14T04:29:58.830 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-14T04:29:58.831 DEBUG:teuthology.orchestra.run.smithi012:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-14T04:29:58.864 DEBUG:teuthology.nuke:shortname: smithi012 2023-07-14T04:29:58.865 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T04:29:58.883 INFO:teuthology.provision.fog.smithi012:Node is ready 2023-07-14T04:29:58.884 DEBUG:teuthology.orchestra.run.smithi012:> hostname 2023-07-14T04:29:58.904 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012.front.sepia.ceph.com 2023-07-14T04:29:58.905 DEBUG:teuthology.orchestra.run.smithi012:> grep smithi012.front.sepia.ceph.com /etc/hosts 2023-07-14T04:29:58.913 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-13_23:26:42-fs-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7336972', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 04:26:19.512036', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAAgmj36rRzvBZ3w2oImBELDWKJRh5enPpRpW5FGfGNSXKLRqW7PgTIQshtlwOMied2tEVDn6BA7eq3ntBXiCBzNc3Lr+UOUZWfabHZnruFJ5YWOUz2fAHE1AzNxSUZ5J5xzu3hFWrlHasfRYIrV72E8j8SUP+FSo+zqlxtrF9Lwsgqz1SSjSKlHFWEgVR/gialm3lNOexO9/dUs7MN5hp8x06vnMWUijR3LUcDsDuymQ5rpusQbWfjVYle55miIg3iqquk8VzPOctQlEUJQS2fKixTnv4p3l2PBXXT5sLn43O78vnHNKw8uTQ7ySapbbKr3McgACSHS1d04zQgjPb4CAwbLUv5Rr3jehr2bt0s26UQk5jtER362iAEl6hMt6SS4xIv2VSC26h5ieXzN8+nkCN4l/Wv3xppywFzyMHBpgWze/4Ot4msjFdYDguC0AoOX/tpYKCzWhBAo2qOzLI4HW9nQGSOBtc7mDuhovjd0w0+sdJY9AQYunaAufb6LU='} 2023-07-14T04:29:58.918 DEBUG:teuthology.nuke:shortname: smithi125 2023-07-14T04:29:58.919 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T04:29:58.947 DEBUG:teuthology.nuke:shortname: smithi177 2023-07-14T04:29:58.948 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T04:29:58.965 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12 smithi012.front.sepia.ceph.com smithi012 2023-07-14T04:29:58.965 DEBUG:teuthology.orchestra.run.smithi012:> sudo hostname smithi012 2023-07-14T04:29:58.972 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-13_23:26:42-fs-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7336972', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 04:26:19.508375', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-07-14T04:29:58.992 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-13_23:26:42-fs-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7336972', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 04:26:19.510159', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPiLSP1i/qcIFIdKALkYNiqnsomBOVfPHQzHwDUdGLhfxT3TwOGrmpkf74l+Ds7dCs/b7Zje8N8cJOELws2iFYo='} 2023-07-14T04:30:04.019 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i -e 's/smithi012.front.sepia.ceph.com/smithi012/g' /etc/hosts 2023-07-14T04:30:04.025 INFO:teuthology.orchestra.console:Power off smithi012 2023-07-14T04:30:04.026 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T04:30:09.046 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:09.077 INFO:teuthology.orchestra.console:Power off smithi177 2023-07-14T04:30:09.078 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T04:30:09.116 INFO:teuthology.orchestra.console:Power off smithi125 2023-07-14T04:30:09.117 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T04:30:09.235 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-14T04:30:09.236 DEBUG:teuthology.orchestra.run.smithi177:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-14T04:30:09.628 INFO:teuthology.provision.fog.smithi177:Node is ready 2023-07-14T04:30:09.629 DEBUG:teuthology.orchestra.run.smithi177:> hostname 2023-07-14T04:30:09.642 INFO:teuthology.orchestra.run.smithi177.stdout:smithi177.front.sepia.ceph.com 2023-07-14T04:30:09.643 DEBUG:teuthology.orchestra.run.smithi177:> grep smithi177.front.sepia.ceph.com /etc/hosts 2023-07-14T04:30:09.696 INFO:teuthology.orchestra.run.smithi177.stdout:172.21.15.177 smithi177.front.sepia.ceph.com smithi177 2023-07-14T04:30:09.696 DEBUG:teuthology.orchestra.run.smithi177:> sudo hostname smithi177 2023-07-14T04:30:10.293 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i -e 's/smithi177.front.sepia.ceph.com/smithi177/g' /etc/hosts 2023-07-14T04:30:10.321 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i -e 's/172.21.15.177/172.21.15.177/g' /etc/hosts 2023-07-14T04:30:10.389 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i -e 's/smithi177.front.sepia.ceph.com/smithi177/g' /etc/hostname 2023-07-14T04:30:10.457 DEBUG:teuthology.orchestra.run.smithi177:> sudo hostname smithi177 2023-07-14T04:30:10.523 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2023-07-14T04:30:10.578 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="CentOS Stream" 2023-07-14T04:30:10.578 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="9" 2023-07-14T04:30:10.578 INFO:teuthology.orchestra.run.smithi177.stdout:ID="centos" 2023-07-14T04:30:10.578 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="rhel fedora" 2023-07-14T04:30:10.578 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="9" 2023-07-14T04:30:10.579 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el9" 2023-07-14T04:30:10.579 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-14T04:30:10.579 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2023-07-14T04:30:10.579 INFO:teuthology.orchestra.run.smithi177.stdout:LOGO="fedora-logo-icon" 2023-07-14T04:30:10.579 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-14T04:30:10.579 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://centos.org/" 2023-07-14T04:30:10.579 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T04:30:10.580 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-14T04:30:10.580 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-14T04:30:10.580 INFO:teuthology.provision.fog.smithi177:Deploy complete! 2023-07-14T04:30:14.058 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:14.103 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:14.143 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:19.116 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:19.154 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:19.168 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:23.270 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:24.227 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:24.279 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:28.383 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:29.340 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:29.391 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:33.496 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:34.452 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:34.502 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:38.609 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:39.564 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:39.611 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:43.719 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:44.675 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:44.720 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:48.830 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:49.040 INFO:teuthology.orchestra.console:Power off for smithi125 completed 2023-07-14T04:30:54.144 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:54.152 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:30:59.223 INFO:teuthology.lock.ops:unlocked: smithi125.front.sepia.ceph.com 2023-07-14T04:30:59.294 INFO:teuthology.orchestra.console:Power off for smithi177 completed 2023-07-14T04:30:59.295 INFO:teuthology.orchestra.console:Power off for smithi012 completed 2023-07-14T04:31:00.540 INFO:teuthology.lock.ops:unlocked: smithi012.front.sepia.ceph.com 2023-07-14T04:31:00.556 INFO:teuthology.lock.ops:unlocked: smithi177.front.sepia.ceph.com 2023-07-14T04:31:00.949 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/dispatcher/supervisor.py", line 57, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi125's OS to be centos 9 but found ubuntu 20.04