2023-07-25T21:28:46.982 INFO:root:teuthology version: 0.0.1.dev160+g407880c 2023-07-25T21:28:46.984 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-25T21:28:46.984 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-25T21:28:47.142 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f40f25073a0>, signals=[15]) 2023-07-25T21:28:47.144 INFO:teuthology.lock.ops:Start node 'smithi105.front.sepia.ceph.com' reimaging 2023-07-25T21:28:47.145 INFO:teuthology.lock.ops:Updating [smithi105.front.sepia.ceph.com]: reset os type and version on server 2023-07-25T21:28:47.145 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2023-07-25T21:28:47.169 INFO:teuthology.lock.ops:Start node 'smithi159.front.sepia.ceph.com' reimaging 2023-07-25T21:28:47.169 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: reset os type and version on server 2023-07-25T21:28:47.170 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2023-07-25T21:28:47.171 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /host {"name": "smithi105"} 2023-07-25T21:28:54.534 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /host {"name": "smithi159"} 2023-07-25T21:29:01.527 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-25T21:29:06.533 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-25T21:29:11.578 DEBUG:teuthology.provision.fog.smithi105:Requesting image smithi_centos_8.stream (ID 3) 2023-07-25T21:29:11.579 DEBUG:teuthology.provision.fog.smithi105:FOG request: PUT /host/210 {"imageID": 3} 2023-07-25T21:29:14.367 DEBUG:teuthology.provision.fog.smithi159:Requesting image smithi_centos_8.stream (ID 3) 2023-07-25T21:29:14.367 DEBUG:teuthology.provision.fog.smithi159:FOG request: PUT /host/264 {"imageID": 3} 2023-07-25T21:29:14.414 INFO:teuthology.provision.fog.smithi159:Scheduling deploy of centos 8.stream 2023-07-25T21:29:14.414 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None 2023-07-25T21:29:14.430 INFO:teuthology.provision.fog.smithi105:Scheduling deploy of centos 8.stream 2023-07-25T21:29:14.431 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-07-25T21:29:14.460 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /tasktype {"name": "deploy"} 2023-07-25T21:29:14.474 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /tasktype {"name": "deploy"} 2023-07-25T21:29:14.497 DEBUG:teuthology.provision.fog.smithi159:FOG request: POST /host/264/task {"taskTypeID": 1} 2023-07-25T21:29:14.511 DEBUG:teuthology.provision.fog.smithi105:FOG request: POST /host/210/task {"taskTypeID": 1} 2023-07-25T21:29:14.689 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None 2023-07-25T21:29:14.695 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-07-25T21:29:24.783 INFO:teuthology.orchestra.console:Power off smithi105 2023-07-25T21:29:24.784 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-25T21:29:24.802 INFO:teuthology.orchestra.console:Power off smithi159 2023-07-25T21:29:24.803 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-25T21:29:24.813 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-25T21:29:24.814 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:29:24.829 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-25T21:29:24.830 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:29:24.841 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-25T21:29:24.857 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-25T21:29:28.844 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:29:28.858 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:29:29.956 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:29:29.969 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:29:33.957 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:29:33.970 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:29:35.069 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:29:35.080 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:29:39.071 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:29:39.081 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:29:40.183 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:29:40.191 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:29:44.188 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:29:44.197 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:29:45.299 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:29:45.308 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:29:49.304 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:29:49.313 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:29:50.415 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:29:50.424 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:29:54.417 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:29:54.426 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:29:55.529 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:29:55.536 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:29:59.531 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:29:59.539 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:30:00.641 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:30:00.648 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:30:04.644 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:30:04.652 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:30:05.756 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:30:05.761 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:30:09.760 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:30:09.768 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:30:10.871 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:30:10.878 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:30:14.873 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:30:14.881 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:30:15.983 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:30:15.991 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:30:19.985 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:30:19.993 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:30:20.105 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-25T21:30:20.206 INFO:teuthology.orchestra.console:Power off for smithi159 completed 2023-07-25T21:30:20.306 INFO:teuthology.orchestra.console:Power on smithi159 2023-07-25T21:30:20.307 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-25T21:30:20.330 DEBUG:teuthology.orchestra.console:power on output: Chassis Power Control: Up/On 2023-07-25T21:30:20.331 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:30:20.351 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-25T21:30:21.096 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:30:24.356 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:30:24.465 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-25T21:30:24.567 INFO:teuthology.orchestra.console:Power on for smithi159 completed 2023-07-25T21:30:24.668 INFO:teuthology.provision.fog.smithi159:Waiting for deploy to finish 2023-07-25T21:30:24.668 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None 2023-07-25T21:30:29.736 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:30:30.847 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:30:34.849 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:30:34.959 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-25T21:30:35.060 INFO:teuthology.orchestra.console:Power off for smithi105 completed 2023-07-25T21:30:35.160 INFO:teuthology.orchestra.console:Power on smithi105 2023-07-25T21:30:35.161 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-25T21:30:40.185 DEBUG:teuthology.orchestra.console:power on output: Chassis Power Control: Up/On 2023-07-25T21:30:40.186 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:30:41.195 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:30:44.804 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None 2023-07-25T21:30:49.813 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:30:50.257 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-25T21:30:50.357 INFO:teuthology.orchestra.console:Power on for smithi105 completed 2023-07-25T21:30:50.458 INFO:teuthology.provision.fog.smithi105:Waiting for deploy to finish 2023-07-25T21:30:50.459 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-07-25T21:30:50.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T21:30:53.588 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-07-25T21:30:59.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T21:31:02.672 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-07-25T21:31:04.887 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None 2023-07-25T21:31:08.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T21:31:16.752 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-07-25T21:31:21.630 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None 2023-07-25T21:31:22.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T21:31:30.832 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-07-25T21:31:36.700 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None 2023-07-25T21:31:41.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T21:31:46.749 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T21:31:49.776 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-07-25T21:31:55.780 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T21:32:18.555 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-25T21:32:18.557 DEBUG:teuthology.orchestra.run.smithi159:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-25T21:32:19.184 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-07-25T21:32:19.407 INFO:teuthology.provision.fog.smithi159:Node is ready 2023-07-25T21:32:19.407 DEBUG:teuthology.orchestra.run.smithi159:> hostname 2023-07-25T21:32:19.455 INFO:teuthology.orchestra.run.smithi159.stdout:smithi159.front.sepia.ceph.com 2023-07-25T21:32:19.455 DEBUG:teuthology.orchestra.run.smithi159:> grep smithi159.front.sepia.ceph.com /etc/hosts 2023-07-25T21:32:19.505 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159 smithi159.front.sepia.ceph.com smithi159 2023-07-25T21:32:19.506 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159 2023-07-25T21:32:19.723 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi159.front.sepia.ceph.com/smithi159/g' /etc/hosts 2023-07-25T21:32:19.736 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/172.21.15.159/172.21.15.159/g' /etc/hosts 2023-07-25T21:32:19.797 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi159.front.sepia.ceph.com/smithi159/g' /etc/hostname 2023-07-25T21:32:19.854 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159 2023-07-25T21:32:19.913 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2023-07-25T21:32:19.962 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="Ubuntu" 2023-07-25T21:32:19.962 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-07-25T21:32:19.962 INFO:teuthology.orchestra.run.smithi159.stdout:ID=ubuntu 2023-07-25T21:32:19.963 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE=debian 2023-07-25T21:32:19.963 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-07-25T21:32:19.963 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="20.04" 2023-07-25T21:32:19.963 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://www.ubuntu.com/" 2023-07-25T21:32:19.963 INFO:teuthology.orchestra.run.smithi159.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-07-25T21:32:19.963 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-07-25T21:32:19.963 INFO:teuthology.orchestra.run.smithi159.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-07-25T21:32:19.964 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_CODENAME=focal 2023-07-25T21:32:19.964 INFO:teuthology.orchestra.run.smithi159.stdout:UBUNTU_CODENAME=focal 2023-07-25T21:32:19.966 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi159's OS to be centos 8 but found ubuntu 20.04 2023-07-25T21:32:19.966 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-25T21:32:19.967 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-25T21:32:22.991 INFO:teuthology.nuke:Checking targets against current locks 2023-07-25T21:32:23.043 DEBUG:teuthology.nuke:shortname: smithi105 2023-07-25T21:32:23.043 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-25T21:32:23.064 DEBUG:teuthology.nuke:shortname: smithi159 2023-07-25T21:32:23.065 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-25T21:32:23.069 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2023-07-25_19:56:53-orch:cephadm-wip-guits-testing-2-2023-07-25-0836-distro-default-smithi/7352019', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-25 21:28:46.135334', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvsqetM4FOKuyCAiIguEMT48dcd7Ih8rr8YEW9IkxGrmj5QZkuIbaA02ai6mScG/2FGvVL+cMoWExOdOSV2bSbyqb0kwYNTy7ebPr+FXTiMEf6WyxwzsPHjWjnsWaMAA9A+IsFK1U3ZbA2kH19n1+MQrxkr+GGQ6o6Wv6yO7TREjH09BOLbE47nDRq/Xxonty77rwogB8e3ue+h8i0KOmfgnq9K9OSAyNNO+wDUlOHyste3C7GluP0uSPW7x1rWzF9cuC0HyajKylVI+trF3nyWeqBbu9OTw4MWwQoMn4X9/CT14witzRaw4zS85TLTI7s0JWFQFT7iv5cKxuYOOPwDUDNvsviwK3HtAIO+aQ4s4TU/WprQZDC/034Pd/qsNFU42bo/AfxZrWspP9TLL/Lb7EKpBEAVnSD6ZOmGp13UtVap0KxjBvdWsDcc+kzXAKANYxwIrKeEt2QoG4CQkOaraJoV9nE3aaMO2X5OR1YGZFfd2UGryEmMwUSpHtJrVk='} 2023-07-25T21:32:23.094 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2023-07-25_19:56:53-orch:cephadm-wip-guits-testing-2-2023-07-25-0836-distro-default-smithi/7352019', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-25 21:28:46.137439', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0='} 2023-07-25T21:32:23.127 INFO:teuthology.orchestra.console:Power off smithi105 2023-07-25T21:32:23.128 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-25T21:32:23.150 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-25T21:32:23.151 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:32:23.166 INFO:teuthology.orchestra.console:Power off smithi159 2023-07-25T21:32:23.167 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-25T21:32:23.180 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-25T21:32:23.202 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-25T21:32:23.203 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:32:23.228 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-25T21:32:25.186 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T21:32:27.182 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:32:27.230 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:32:28.240 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-07-25T21:32:28.292 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:32:28.340 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:32:32.294 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:32:32.341 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:32:33.406 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:32:33.451 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:32:34.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T21:32:37.296 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-07-25T21:32:37.407 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:32:37.452 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:32:37.519 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-25T21:32:37.620 INFO:teuthology.orchestra.console:Power off for smithi105 completed 2023-07-25T21:32:41.742 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:32:46.748 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T21:32:46.750 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:32:46.781 INFO:teuthology.lock.ops:unlocked: smithi105.front.sepia.ceph.com 2023-07-25T21:32:47.859 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:32:49.808 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-07-25T21:32:51.862 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:32:52.972 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:32:55.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T21:33:00.817 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:33:01.928 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:33:03.892 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-07-25T21:33:05.930 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:33:07.040 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:33:09.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T21:33:14.900 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:33:16.009 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:33:17.968 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-07-25T21:33:20.012 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:33:21.122 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T21:33:23.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T21:33:28.979 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T21:33:29.931 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-25T21:33:30.032 INFO:teuthology.orchestra.console:Power off for smithi159 completed 2023-07-25T21:33:35.141 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-07-25T21:33:35.205 INFO:teuthology.lock.ops:unlocked: smithi159.front.sepia.ceph.com 2023-07-25T21:33:35.476 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/supervisor.py", line 57, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi159's OS to be centos 8 but found ubuntu 20.04