2023-07-27T09:45:06.012 INFO:root:teuthology version: 0.0.1.dev160+g407880c 2023-07-27T09:45:06.014 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T09:45:06.014 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T09:45:06.171 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1437ff3310>, signals=[15]) 2023-07-27T09:45:06.172 INFO:teuthology.lock.ops:Start node 'smithi046.front.sepia.ceph.com' reimaging 2023-07-27T09:45:06.172 INFO:teuthology.lock.ops:Updating [smithi046.front.sepia.ceph.com]: reset os type and version on server 2023-07-27T09:45:06.173 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2023-07-27T09:45:06.202 INFO:teuthology.lock.ops:Start node 'smithi145.front.sepia.ceph.com' reimaging 2023-07-27T09:45:06.203 INFO:teuthology.lock.ops:Updating [smithi145.front.sepia.ceph.com]: reset os type and version on server 2023-07-27T09:45:06.203 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2023-07-27T09:45:06.204 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /host {"name": "smithi046"} 2023-07-27T09:45:16.234 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /host {"name": "smithi145"} 2023-07-27T09:45:24.185 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-07-27T09:45:34.233 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-07-27T09:45:38.302 DEBUG:teuthology.provision.fog.smithi046:Requesting image smithi_rhel_8.4 (ID 7) 2023-07-27T09:45:38.303 DEBUG:teuthology.provision.fog.smithi046:FOG request: PUT /host/151 {"imageID": 7} 2023-07-27T09:45:38.418 DEBUG:teuthology.provision.fog.smithi145:Requesting image smithi_rhel_8.4 (ID 7) 2023-07-27T09:45:38.418 DEBUG:teuthology.provision.fog.smithi145:FOG request: PUT /host/250 {"imageID": 7} 2023-07-27T09:45:38.528 INFO:teuthology.provision.fog.smithi046:Scheduling deploy of rhel 8.4 2023-07-27T09:45:38.528 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None 2023-07-27T09:45:43.534 INFO:teuthology.provision.fog.smithi145:Scheduling deploy of rhel 8.4 2023-07-27T09:45:43.534 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-27T09:45:53.638 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /tasktype {"name": "deploy"} 2023-07-27T09:45:53.651 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /tasktype {"name": "deploy"} 2023-07-27T09:45:53.704 DEBUG:teuthology.provision.fog.smithi145:FOG request: POST /host/250/task {"taskTypeID": 1} 2023-07-27T09:45:53.722 DEBUG:teuthology.provision.fog.smithi046:FOG request: POST /host/151/task {"taskTypeID": 1} 2023-07-27T09:45:53.819 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-27T09:45:53.822 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None 2023-07-27T09:46:03.928 INFO:teuthology.orchestra.console:Power off smithi046 2023-07-27T09:46:03.929 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-27T09:46:03.947 INFO:teuthology.orchestra.console:Power off smithi145 2023-07-27T09:46:03.947 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-27T09:46:08.956 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-27T09:46:08.957 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:08.975 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-27T09:46:08.976 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:08.986 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T09:46:09.986 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:12.988 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:13.987 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:14.099 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:15.097 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:18.101 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:19.100 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:19.210 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:20.212 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:23.215 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:24.213 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:24.325 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:25.325 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:28.327 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:29.327 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:29.437 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:30.437 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:33.440 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:34.439 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:34.550 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:35.550 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:38.552 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:39.550 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:39.662 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:40.660 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:43.664 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:44.661 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:44.772 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:45.771 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:48.777 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:49.772 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:49.885 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:50.883 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:53.888 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:54.885 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:54.995 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T09:46:54.997 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:46:55.096 INFO:teuthology.orchestra.console:Power off for smithi145 completed 2023-07-27T09:46:55.197 INFO:teuthology.orchestra.console:Power on smithi145 2023-07-27T09:46:55.197 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-27T09:46:55.217 DEBUG:teuthology.orchestra.console:power on output: Chassis Power Control: Up/On 2023-07-27T09:46:55.218 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:55.238 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T09:46:59.000 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:59.240 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:46:59.350 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T09:47:00.110 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:47:03.352 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:47:04.112 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:47:04.462 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:47:05.225 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:47:08.464 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:47:08.574 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T09:47:08.675 INFO:teuthology.orchestra.console:Power on for smithi145 completed 2023-07-27T09:47:08.776 INFO:teuthology.provision.fog.smithi145:Waiting for deploy to finish 2023-07-27T09:47:08.776 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-27T09:47:13.784 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:47:13.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T09:47:14.893 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:47:16.944 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2023-07-27T09:47:18.896 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:47:20.007 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:47:22.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T09:47:24.010 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:47:25.121 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:47:25.122 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds 2023-07-27T09:47:25.123 ERROR:teuthology.orchestra.console:Failed to power off smithi046 2023-07-27T09:47:25.123 INFO:teuthology.orchestra.console:Power on smithi046 2023-07-27T09:47:25.124 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-27T09:47:26.000 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2023-07-27T09:47:32.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T09:47:37.015 DEBUG:teuthology.orchestra.console:power on output: Chassis Power Control: Up/On 2023-07-27T09:47:37.016 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:47:38.024 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:47:40.085 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2023-07-27T09:47:42.027 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:47:42.137 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T09:47:42.238 INFO:teuthology.orchestra.console:Power on for smithi046 completed 2023-07-27T09:47:42.339 INFO:teuthology.provision.fog.smithi046:Waiting for deploy to finish 2023-07-27T09:47:42.339 DEBUG:teuthology.provision.fog.smithi046:FOG request: GET /task/active None 2023-07-27T09:47:47.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T09:47:52.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T09:47:57.396 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2023-07-27T09:48:00.464 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46 2023-07-27T09:48:03.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T09:48:06.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T09:48:06.484 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2023-07-27T09:48:09.520 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46 2023-07-27T09:48:12.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T09:48:17.495 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T09:48:25.552 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46 2023-07-27T09:48:31.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T09:48:34.608 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46 2023-07-27T09:48:35.824 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2023-07-27T09:48:40.612 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T09:48:45.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T09:48:51.003 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-27T09:48:51.005 DEBUG:teuthology.orchestra.run.smithi145:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-27T09:48:51.288 INFO:teuthology.provision.fog.smithi145:Node is ready 2023-07-27T09:48:51.289 DEBUG:teuthology.orchestra.run.smithi145:> hostname 2023-07-27T09:48:51.306 INFO:teuthology.orchestra.run.smithi145.stdout:smithi145.front.sepia.ceph.com 2023-07-27T09:48:51.307 DEBUG:teuthology.orchestra.run.smithi145:> grep smithi145.front.sepia.ceph.com /etc/hosts 2023-07-27T09:48:51.369 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145 smithi145.front.sepia.ceph.com smithi145 2023-07-27T09:48:51.370 DEBUG:teuthology.orchestra.run.smithi145:> sudo hostname smithi145 2023-07-27T09:48:51.504 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i -e 's/smithi145.front.sepia.ceph.com/smithi145/g' /etc/hosts 2023-07-27T09:48:51.532 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i -e 's/172.21.15.145/172.21.15.145/g' /etc/hosts 2023-07-27T09:48:51.600 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i -e 's/smithi145.front.sepia.ceph.com/smithi145/g' /etc/hostname 2023-07-27T09:48:51.672 DEBUG:teuthology.orchestra.run.smithi145:> sudo hostname smithi145 2023-07-27T09:48:51.740 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2023-07-27T09:48:51.802 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="CentOS Stream" 2023-07-27T09:48:51.802 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="8" 2023-07-27T09:48:51.802 INFO:teuthology.orchestra.run.smithi145.stdout:ID="centos" 2023-07-27T09:48:51.803 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="rhel fedora" 2023-07-27T09:48:51.803 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="8" 2023-07-27T09:48:51.803 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el8" 2023-07-27T09:48:51.803 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-27T09:48:51.803 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2023-07-27T09:48:51.803 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-27T09:48:51.803 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://centos.org/" 2023-07-27T09:48:51.804 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-27T09:48:51.804 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-27T09:48:51.804 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-27T09:48:51.806 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi145's OS to be rhel 8.4 but found centos 8 2023-07-27T09:48:51.806 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T09:48:51.807 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T09:48:56.897 INFO:teuthology.nuke:Checking targets against current locks 2023-07-27T09:49:01.687 DEBUG:teuthology.nuke:shortname: smithi046 2023-07-27T09:49:01.687 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-27T09:49:01.705 DEBUG:teuthology.nuke:shortname: smithi145 2023-07-27T09:49:01.705 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-27T09:49:01.711 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-26_15:58:31-rados-wip-yuri8-testing-2023-07-24-0819-quincy-distro-default-smithi/7353925', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-27 09:45:05.139112', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6nCOgGw6jCJZ18M1O7mXnPg4K28myjeOlT8dbeuWxAC9TNfSOEYn6hy1LjO2bCuoCVPykxxFE9Vxmyb+48WvjKIdCIAnlft4eMhtnqL0qN5Kvvm7tEChfa6fTep3GQUTXKDSBepb7RtIZ3HTJ4DZXCKYPUBHpVnpTBVu4oGsvJk1s8wB7zcNnSMPZc9pOZadT4/X0Si68X19IfxhlZd/63tpCcEoxKgcQEGNGFBEh//35YX1al9YummM+ZZFkB5HA8nUTPd8rpIsuLSe1XNQ1S7+WQVyyaOaDjV3UjyD33lpWgJbDQk6OXFdwnXP/od67SX51/RJtlca+7KVEyb+64tlenPlZ7tlnbMO4pBncrJqOT1gdiO414Ao1uwu2Xc/QPIrBBXDSWHFnPOuwYfb7oQ/mNNEO8gydDwWRa3Mm+xxHrH5F51hgiIfzpjyCrh+ERVenSi1J65ks0Wdi3FZBXHlJCM0bn2ktaqyHVThZN+lIAapLtdty76UgnA4PHa8='} 2023-07-27T09:49:01.724 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-26_15:58:31-rados-wip-yuri8-testing-2023-07-24-0819-quincy-distro-default-smithi/7353925', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-27 09:45:05.137945', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNK65MmjoTUZlZtVxh+WtVGLnFbZKdmvOhXQ3PCj6rTasQeQohPproUkj3q9wOzhsn/uQeMwyBArbq4dVEc72XM='} 2023-07-27T09:49:01.755 INFO:teuthology.orchestra.console:Power off smithi046 2023-07-27T09:49:01.756 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-27T09:49:01.777 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-27T09:49:01.778 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:49:01.799 INFO:teuthology.orchestra.console:Power off smithi145 2023-07-27T09:49:01.800 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-27T09:49:01.810 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T09:49:01.822 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-27T09:49:01.823 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:49:01.844 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T09:49:05.812 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:49:05.845 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:49:05.958 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T09:49:06.923 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:49:09.960 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:49:10.925 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:49:11.070 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:49:12.034 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:49:15.072 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:49:16.036 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:49:16.183 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:49:17.148 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:49:20.185 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:49:21.150 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:49:21.295 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:49:22.262 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:49:25.299 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:49:26.262 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:49:26.410 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:49:27.374 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:49:30.414 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:49:31.376 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:49:31.524 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:49:32.490 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:49:35.526 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:49:35.636 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T09:49:35.737 INFO:teuthology.orchestra.console:Power off for smithi145 completed 2023-07-27T09:49:40.846 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:49:45.885 WARNING:teuthology.provision.fog.smithi046:timed out 2023-07-27T09:49:45.918 INFO:teuthology.lock.ops:unlocked: smithi145.front.sepia.ceph.com 2023-07-27T09:49:46.889 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T09:49:50.890 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T09:49:51.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T09:49:56.894 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T09:49:56.995 INFO:teuthology.orchestra.console:Power off for smithi046 completed 2023-07-27T09:50:07.156 INFO:teuthology.lock.ops:unlocked: smithi046.front.sepia.ceph.com 2023-07-27T09:50:17.285 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46 2023-07-27T09:50:17.421 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/supervisor.py", line 57, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi145's OS to be rhel 8.4 but found centos 8