2023-07-14T02:08:31.044 INFO:root:teuthology version: 0.0.1.dev143+g073e6eb 2023-07-14T02:08:31.046 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T02:08:31.047 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T02:08:31.219 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f103be41f70>, signals=[15]) 2023-07-14T02:08:31.222 INFO:teuthology.lock.ops:Start node 'smithi042.front.sepia.ceph.com' reimaging 2023-07-14T02:08:31.223 INFO:teuthology.lock.ops:Updating [smithi042.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T02:08:31.223 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2023-07-14T02:08:31.252 INFO:teuthology.lock.ops:Start node 'smithi122.front.sepia.ceph.com' reimaging 2023-07-14T02:08:31.253 INFO:teuthology.lock.ops:Updating [smithi122.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T02:08:31.254 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2023-07-14T02:08:31.256 DEBUG:teuthology.provision.fog.smithi042:FOG request: GET /host {"name": "smithi042"} 2023-07-14T02:08:34.602 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /host {"name": "smithi122"} 2023-07-14T02:08:44.274 DEBUG:teuthology.provision.fog.smithi042:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-14T02:08:54.321 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-14T02:08:59.331 DEBUG:teuthology.provision.fog.smithi042:Requesting image smithi_centos_8.stream (ID 3) 2023-07-14T02:08:59.331 DEBUG:teuthology.provision.fog.smithi042:FOG request: PUT /host/147 {"imageID": 3} 2023-07-14T02:08:59.944 DEBUG:teuthology.provision.fog.smithi122:Requesting image smithi_centos_8.stream (ID 3) 2023-07-14T02:08:59.944 DEBUG:teuthology.provision.fog.smithi122:FOG request: PUT /host/227 {"imageID": 3} 2023-07-14T02:08:59.958 INFO:teuthology.provision.fog.smithi042:Scheduling deploy of centos 8.stream 2023-07-14T02:08:59.958 DEBUG:teuthology.provision.fog.smithi042:FOG request: GET /task/active None 2023-07-14T02:08:59.993 INFO:teuthology.provision.fog.smithi122:Scheduling deploy of centos 8.stream 2023-07-14T02:08:59.994 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None 2023-07-14T02:09:00.028 DEBUG:teuthology.provision.fog.smithi042:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T02:09:00.048 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T02:09:00.065 DEBUG:teuthology.provision.fog.smithi042:FOG request: POST /host/147/task {"taskTypeID": 1} 2023-07-14T02:09:00.087 DEBUG:teuthology.provision.fog.smithi122:FOG request: POST /host/227/task {"taskTypeID": 1} 2023-07-14T02:09:00.128 DEBUG:teuthology.provision.fog.smithi042:FOG request: GET /task/active None 2023-07-14T02:09:00.207 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None 2023-07-14T02:09:10.305 INFO:teuthology.orchestra.console:Power off smithi122 2023-07-14T02:09:10.306 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T02:09:10.330 INFO:teuthology.orchestra.console:Power off smithi042 2023-07-14T02:09:10.331 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T02:09:10.345 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:10.357 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:14.376 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:14.386 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:19.489 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:19.501 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:24.603 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:24.613 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:29.719 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:29.731 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:34.835 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:34.920 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:40.029 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:40.038 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:45.141 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:45.152 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:50.256 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:50.267 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:55.370 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:09:55.385 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:00.492 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:00.507 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:05.614 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:05.627 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:10.731 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:10.747 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:15.844 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:15.860 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:20.956 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:20.975 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:22.071 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds 2023-07-14T02:10:22.072 ERROR:teuthology.orchestra.console:Failed to power off smithi122 2023-07-14T02:10:22.072 INFO:teuthology.orchestra.console:Power on smithi122 2023-07-14T02:10:22.072 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T02:10:22.086 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds 2023-07-14T02:10:22.087 ERROR:teuthology.orchestra.console:Failed to power off smithi042 2023-07-14T02:10:22.087 INFO:teuthology.orchestra.console:Power on smithi042 2023-07-14T02:10:22.088 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T02:10:30.947 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:34.974 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:37.103 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:40.089 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:42.117 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:45.203 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:45.414 INFO:teuthology.orchestra.console:Power on for smithi042 completed 2023-07-14T02:10:45.515 INFO:teuthology.provision.fog.smithi042:Waiting for deploy to finish 2023-07-14T02:10:45.515 DEBUG:teuthology.provision.fog.smithi042:FOG request: GET /task/active None 2023-07-14T02:10:47.231 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:10:47.442 INFO:teuthology.orchestra.console:Power on for smithi122 completed 2023-07-14T02:10:47.543 INFO:teuthology.provision.fog.smithi122:Waiting for deploy to finish 2023-07-14T02:10:47.544 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None 2023-07-14T02:10:47.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T02:10:50.676 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-07-14T02:10:56.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T02:11:06.689 DEBUG:teuthology.provision.fog.smithi042:FOG request: GET /task/active None 2023-07-14T02:11:11.697 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-07-14T02:11:11.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T02:11:14.800 WARNING:teuthology.provision.fog.smithi042:[Errno None] Unable to connect to port 22 on 172.21.15.42 2023-07-14T02:11:17.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T02:11:22.705 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T02:11:25.776 WARNING:teuthology.provision.fog.smithi042:[Errno None] Unable to connect to port 22 on 172.21.15.42 2023-07-14T02:11:25.780 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-07-14T02:11:31.785 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T02:11:31.788 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T02:11:36.793 WARNING:teuthology.provision.fog.smithi042:[Errno None] Unable to connect to port 22 on 172.21.15.42 2023-07-14T02:11:39.856 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-07-14T02:11:42.797 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T02:11:47.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T02:12:19.757 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-14T02:12:19.758 DEBUG:teuthology.orchestra.run.smithi042:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-14T02:12:20.119 INFO:teuthology.provision.fog.smithi042:Node is ready 2023-07-14T02:12:20.119 DEBUG:teuthology.orchestra.run.smithi042:> hostname 2023-07-14T02:12:20.138 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042.front.sepia.ceph.com 2023-07-14T02:12:20.139 DEBUG:teuthology.orchestra.run.smithi042:> grep smithi042.front.sepia.ceph.com /etc/hosts 2023-07-14T02:12:20.193 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42 smithi042.front.sepia.ceph.com smithi042 2023-07-14T02:12:20.193 DEBUG:teuthology.orchestra.run.smithi042:> sudo hostname smithi042 2023-07-14T02:12:20.341 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i -e 's/smithi042.front.sepia.ceph.com/smithi042/g' /etc/hosts 2023-07-14T02:12:20.370 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i -e 's/172.21.15.42/172.21.15.42/g' /etc/hosts 2023-07-14T02:12:20.444 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i -e 's/smithi042.front.sepia.ceph.com/smithi042/g' /etc/hostname 2023-07-14T02:12:20.520 DEBUG:teuthology.orchestra.run.smithi042:> sudo hostname smithi042 2023-07-14T02:12:20.591 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2023-07-14T02:12:20.652 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T02:12:20.652 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T02:12:20.653 INFO:teuthology.orchestra.run.smithi042.stdout:ID="rhel" 2023-07-14T02:12:20.653 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="fedora" 2023-07-14T02:12:20.653 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="8.6" 2023-07-14T02:12:20.653 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el8" 2023-07-14T02:12:20.653 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T02:12:20.653 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2023-07-14T02:12:20.654 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T02:12:20.654 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T02:12:20.654 INFO:teuthology.orchestra.run.smithi042.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T02:12:20.654 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T02:12:20.654 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-07-14T02:12:20.654 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T02:12:20.654 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T02:12:20.654 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T02:12:20.655 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T02:12:20.656 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi042's OS to be centos 8 but found rhel 8.6 2023-07-14T02:12:20.657 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T02:12:20.658 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T02:12:25.735 INFO:teuthology.nuke:Checking targets against current locks 2023-07-14T02:12:26.923 DEBUG:teuthology.nuke:shortname: smithi042 2023-07-14T02:12:26.924 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T02:12:26.941 DEBUG:teuthology.nuke:shortname: smithi122 2023-07-14T02:12:26.941 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T02:12:26.945 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-13_23:12:00-rados-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7336611', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 02:08:30.148286', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-14T02:12:26.970 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-13_23:12:00-rados-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7336611', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 02:08:30.150325', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-14T02:12:27.003 INFO:teuthology.orchestra.console:Power off smithi042 2023-07-14T02:12:27.003 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T02:12:27.022 INFO:teuthology.orchestra.console:Power off smithi122 2023-07-14T02:12:27.022 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T02:12:27.034 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:12:27.050 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:12:31.062 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:12:31.075 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:12:36.174 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:12:36.189 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T02:12:36.387 INFO:teuthology.orchestra.console:Power off for smithi042 completed 2023-07-14T02:12:46.535 INFO:teuthology.orchestra.console:Power off for smithi122 completed 2023-07-14T02:12:46.564 INFO:teuthology.lock.ops:unlocked: smithi042.front.sepia.ceph.com 2023-07-14T02:12:48.806 INFO:teuthology.lock.ops:unlocked: smithi122.front.sepia.ceph.com 2023-07-14T02:12:49.292 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/dispatcher/supervisor.py", line 57, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi042's OS to be centos 8 but found rhel 8.6