2023-07-28T18:16:19.846 INFO:root:teuthology version: 0.0.1.dev165+g3117e89 2023-07-28T18:16:19.848 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-28T18:16:19.848 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-28T18:16:44.493 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f72edb06310>, signals=[15]) 2023-07-28T18:16:44.495 INFO:teuthology.lock.ops:Start node 'smithi007.front.sepia.ceph.com' reimaging 2023-07-28T18:16:44.495 INFO:teuthology.lock.ops:Updating [smithi007.front.sepia.ceph.com]: reset os type and version on server 2023-07-28T18:16:44.495 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2023-07-28T18:16:46.551 INFO:teuthology.lock.ops:Start node 'smithi139.front.sepia.ceph.com' reimaging 2023-07-28T18:16:46.552 INFO:teuthology.lock.ops:Updating [smithi139.front.sepia.ceph.com]: reset os type and version on server 2023-07-28T18:16:46.552 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2023-07-28T18:16:46.553 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /host {"name": "smithi007"} 2023-07-28T18:16:56.591 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /host {"name": "smithi139"} 2023-07-28T18:17:01.921 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-28T18:17:11.963 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-28T18:17:16.970 DEBUG:teuthology.provision.fog.smithi007:Requesting image smithi_centos_8.stream (ID 3) 2023-07-28T18:17:16.970 DEBUG:teuthology.provision.fog.smithi007:FOG request: PUT /host/113 {"imageID": 3} 2023-07-28T18:17:17.025 DEBUG:teuthology.provision.fog.smithi139:Requesting image smithi_centos_8.stream (ID 3) 2023-07-28T18:17:17.025 DEBUG:teuthology.provision.fog.smithi139:FOG request: PUT /host/244 {"imageID": 3} 2023-07-28T18:17:17.028 INFO:teuthology.provision.fog.smithi007:Scheduling deploy of centos 8.stream 2023-07-28T18:17:17.028 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None 2023-07-28T18:17:17.069 INFO:teuthology.provision.fog.smithi139:Scheduling deploy of centos 8.stream 2023-07-28T18:17:17.070 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-28T18:17:17.100 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /tasktype {"name": "deploy"} 2023-07-28T18:17:17.132 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /tasktype {"name": "deploy"} 2023-07-28T18:17:17.136 DEBUG:teuthology.provision.fog.smithi007:FOG request: POST /host/113/task {"taskTypeID": 1} 2023-07-28T18:17:17.178 DEBUG:teuthology.provision.fog.smithi139:FOG request: POST /host/244/task {"taskTypeID": 1} 2023-07-28T18:17:17.193 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None 2023-07-28T18:17:17.230 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-28T18:17:27.322 INFO:teuthology.orchestra.console.smithi007:Power off 2023-07-28T18:17:27.323 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-28T18:17:27.338 INFO:teuthology.orchestra.console.smithi139:Power off 2023-07-28T18:17:27.338 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-28T18:17:27.347 DEBUG:teuthology.orchestra.console.smithi007:power off output: Chassis Power Control: Down/Off 2023-07-28T18:17:27.348 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:17:27.359 DEBUG:teuthology.orchestra.console.smithi139:power off output: Chassis Power Control: Down/Off 2023-07-28T18:17:27.360 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:17:27.372 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2023-07-28T18:17:27.385 DEBUG:teuthology.orchestra.console.smithi139:check power output: Chassis Power is on 2023-07-28T18:17:31.376 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:17:31.386 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:17:32.488 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:17:32.496 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:17:36.489 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:17:36.499 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:17:37.602 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:17:37.610 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:17:41.604 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:17:41.612 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:17:42.715 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:17:42.722 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:17:46.717 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:17:46.724 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:17:47.827 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:17:47.834 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:17:51.832 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:17:51.839 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:17:52.942 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:17:52.950 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:17:56.943 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:17:56.951 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:17:58.055 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:17:58.062 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:18:02.058 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:18:02.066 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:18:02.168 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2023-07-28T18:18:02.270 INFO:teuthology.orchestra.console.smithi007:Power off completed 2023-07-28T18:18:02.371 INFO:teuthology.orchestra.console.smithi007:Power on 2023-07-28T18:18:02.371 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-28T18:18:03.177 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:18:07.180 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:18:07.399 DEBUG:teuthology.orchestra.console.smithi007:power on output: Chassis Power Control: Up/On 2023-07-28T18:18:07.399 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:18:08.291 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:18:08.409 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:18:12.296 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:18:12.410 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:18:13.409 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:18:13.520 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:18:17.411 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:18:17.521 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:18:18.522 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:18:18.632 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:18:22.523 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:18:22.633 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:18:22.645 DEBUG:teuthology.orchestra.console.smithi139:check power output: Chassis Power is off 2023-07-28T18:18:22.928 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2023-07-28T18:18:23.027 INFO:teuthology.orchestra.console.smithi139:Power off completed 2023-07-28T18:18:23.033 INFO:teuthology.orchestra.console.smithi007:Power on completed 2023-07-28T18:18:23.129 INFO:teuthology.orchestra.console.smithi139:Power on 2023-07-28T18:18:23.129 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-28T18:18:23.137 INFO:teuthology.provision.fog.smithi007:Waiting for deploy to finish 2023-07-28T18:18:23.139 DEBUG:teuthology.provision.fog.smithi007:FOG request: GET /task/active None 2023-07-28T18:18:27.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-28T18:18:30.450 WARNING:teuthology.provision.fog.smithi007:[Errno None] Unable to connect to port 22 on 172.21.15.7 2023-07-28T18:18:36.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-28T18:18:41.462 DEBUG:teuthology.orchestra.console.smithi139:power on output: Chassis Power Control: Up/On 2023-07-28T18:18:41.462 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:18:42.471 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:18:44.528 WARNING:teuthology.provision.fog.smithi007:[Errno None] Unable to connect to port 22 on 172.21.15.7 2023-07-28T18:18:46.475 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:18:47.586 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:18:50.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-28T18:18:55.537 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:18:56.647 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:18:58.608 WARNING:teuthology.provision.fog.smithi007:[Errno None] Unable to connect to port 22 on 172.21.15.7 2023-07-28T18:19:00.651 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:19:01.760 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:19:04.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-28T18:19:05.762 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:19:06.872 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:19:08.752 WARNING:teuthology.provision.fog.smithi007:[Errno None] Unable to connect to port 22 on 172.21.15.7 2023-07-28T18:19:10.873 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:19:10.983 DEBUG:teuthology.orchestra.console.smithi139:check power output: Chassis Power is on 2023-07-28T18:19:11.085 INFO:teuthology.orchestra.console.smithi139:Power on completed 2023-07-28T18:19:11.185 INFO:teuthology.provision.fog.smithi139:Waiting for deploy to finish 2023-07-28T18:19:11.186 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-28T18:19:11.252 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-28T18:19:14.320 WARNING:teuthology.provision.fog.smithi139:[Errno None] Unable to connect to port 22 on 172.21.15.139 2023-07-28T18:19:14.754 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-28T18:19:20.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-28T18:19:28.404 WARNING:teuthology.provision.fog.smithi139:[Errno None] Unable to connect to port 22 on 172.21.15.139 2023-07-28T18:19:34.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-28T18:19:39.060 WARNING:teuthology.provision.fog.smithi139:[Errno None] Unable to connect to port 22 on 172.21.15.139 2023-07-28T18:19:45.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-28T18:19:53.140 WARNING:teuthology.provision.fog.smithi139:[Errno None] Unable to connect to port 22 on 172.21.15.139 2023-07-28T18:19:59.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-28T18:20:14.762 WARNING:teuthology.provision.fog.smithi007:timed out 2023-07-28T18:20:20.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-28T18:20:26.034 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-28T18:20:26.035 DEBUG:teuthology.orchestra.run.smithi007:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-28T18:20:26.302 INFO:teuthology.provision.fog.smithi007:Node is ready 2023-07-28T18:20:26.302 DEBUG:teuthology.orchestra.run.smithi007:> hostname 2023-07-28T18:20:26.318 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007.front.sepia.ceph.com 2023-07-28T18:20:26.318 DEBUG:teuthology.orchestra.run.smithi007:> grep smithi007.front.sepia.ceph.com /etc/hosts 2023-07-28T18:20:26.374 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7 smithi007.front.sepia.ceph.com smithi007 2023-07-28T18:20:26.375 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname smithi007 2023-07-28T18:20:26.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i -e 's/smithi007.front.sepia.ceph.com/smithi007/g' /etc/hosts 2023-07-28T18:20:26.530 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i -e 's/172.21.15.7/172.21.15.7/g' /etc/hosts 2023-07-28T18:20:26.596 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i -e 's/smithi007.front.sepia.ceph.com/smithi007/g' /etc/hostname 2023-07-28T18:20:26.662 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname smithi007 2023-07-28T18:20:26.728 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2023-07-28T18:20:26.783 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="Red Hat Enterprise Linux" 2023-07-28T18:20:26.784 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="8.4 (Ootpa)" 2023-07-28T18:20:26.784 INFO:teuthology.orchestra.run.smithi007.stdout:ID="rhel" 2023-07-28T18:20:26.784 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="fedora" 2023-07-28T18:20:26.784 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="8.4" 2023-07-28T18:20:26.784 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el8" 2023-07-28T18:20:26.785 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-07-28T18:20:26.785 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2023-07-28T18:20:26.785 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-07-28T18:20:26.785 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://www.redhat.com/" 2023-07-28T18:20:26.785 INFO:teuthology.orchestra.run.smithi007.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-28T18:20:26.785 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-28T18:20:26.786 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-07-28T18:20:26.786 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-28T18:20:26.786 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-07-28T18:20:26.786 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-28T18:20:26.786 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-07-28T18:20:26.788 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi007's OS to be centos 8 but found rhel 8.4 2023-07-28T18:20:26.789 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-28T18:20:26.789 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-28T18:20:31.878 INFO:teuthology.nuke:Checking targets against current locks 2023-07-28T18:20:40.785 DEBUG:teuthology.nuke:shortname: smithi007 2023-07-28T18:20:40.786 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-28T18:20:40.814 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-28_14:25:29-rados-wip-yuri7-testing-2023-07-27-1336-quincy-distro-default-smithi/7355457', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-28 18:16:14.008723', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCglor46souURgC4LSKTshsaMls0qusFONVK6lCkqUrqjNhk/4Z6DhmN8tsQIDuOzB6raGshpRRAkjAkKcZXGdE8F98WdyzvUFYvalQX0gnV1BF1zlfXByBoETJW07Zu5vsQtX7CGrCBGphostRPsXID7YCkrgMz61a7toMICQYzMknpqzxsp5dFdu2Fl70Q59nbSOJhHuwRGf1qF6zwgDrc522nwK8n8DuzEWKZkdFsECpNlCiU2p6yRGUCFJTOHQf614L13GnGkCXhqYoSV1MBEfhGcvrgNeEkadxvcKY+vScwGwPotpP80Mh8ipd8vXsU+UG86oVmxqF5CXJbrH6bjxFbdwhuIhqo6NgIkimjgVD+I92vnRTubAJxarrcsg33VEEc8mHXP7VJqjxlztTElACuSGhB+Z1YY3y7mFctylcFDvph9u+wXTsmaVEkx917Jg/f571JKksaCeJGtfjterASRyj2KK3zLVeEH1vmYiB8oHd9MeOjSMafMqyvvM='} 2023-07-28T18:20:40.820 DEBUG:teuthology.nuke:shortname: smithi139 2023-07-28T18:20:40.821 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-28T18:20:40.840 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-28_14:25:29-rados-wip-yuri7-testing-2023-07-27-1336-quincy-distro-default-smithi/7355457', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-28 18:16:14.012075', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0='} 2023-07-28T18:20:40.864 INFO:teuthology.orchestra.console.smithi007:Power off 2023-07-28T18:20:40.865 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-28T18:20:40.889 DEBUG:teuthology.orchestra.console.smithi007:power off output: Chassis Power Control: Down/Off 2023-07-28T18:20:40.890 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:20:40.908 INFO:teuthology.orchestra.console.smithi139:Power off 2023-07-28T18:20:40.909 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-28T18:20:40.920 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-28T18:20:40.921 DEBUG:teuthology.orchestra.run.smithi139:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-28T18:20:40.922 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2023-07-28T18:20:40.934 DEBUG:teuthology.orchestra.console.smithi139:power off output: Chassis Power Control: Down/Off 2023-07-28T18:20:40.935 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:20:40.957 DEBUG:teuthology.orchestra.console.smithi139:check power output: Chassis Power is on 2023-07-28T18:20:41.437 INFO:teuthology.provision.fog.smithi139:Node is ready 2023-07-28T18:20:41.438 DEBUG:teuthology.orchestra.run.smithi139:> hostname 2023-07-28T18:20:41.445 INFO:teuthology.orchestra.run.smithi139.stdout:smithi139.front.sepia.ceph.com 2023-07-28T18:20:41.446 DEBUG:teuthology.orchestra.run.smithi139:> grep smithi139.front.sepia.ceph.com /etc/hosts 2023-07-28T18:20:41.493 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139 smithi139.front.sepia.ceph.com smithi139 2023-07-28T18:20:41.494 DEBUG:teuthology.orchestra.run.smithi139:> sudo hostname smithi139 2023-07-28T18:20:41.602 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i -e 's/smithi139.front.sepia.ceph.com/smithi139/g' /etc/hosts 2023-07-28T18:20:41.620 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i -e 's/172.21.15.139/172.21.15.139/g' /etc/hosts 2023-07-28T18:20:41.680 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i -e 's/smithi139.front.sepia.ceph.com/smithi139/g' /etc/hostname 2023-07-28T18:20:41.691 DEBUG:teuthology.orchestra.run.smithi139:> sudo hostname smithi139 2023-07-28T18:20:41.752 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2023-07-28T18:20:41.760 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="Ubuntu" 2023-07-28T18:20:41.760 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-07-28T18:20:41.760 INFO:teuthology.orchestra.run.smithi139.stdout:ID=ubuntu 2023-07-28T18:20:41.761 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE=debian 2023-07-28T18:20:41.761 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-07-28T18:20:41.761 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="20.04" 2023-07-28T18:20:41.762 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://www.ubuntu.com/" 2023-07-28T18:20:41.762 INFO:teuthology.orchestra.run.smithi139.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-07-28T18:20:41.762 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-07-28T18:20:41.762 INFO:teuthology.orchestra.run.smithi139.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-07-28T18:20:41.763 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_CODENAME=focal 2023-07-28T18:20:41.763 INFO:teuthology.orchestra.run.smithi139.stdout:UBUNTU_CODENAME=focal 2023-07-28T18:20:44.923 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:20:44.958 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:20:46.034 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:20:46.069 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:20:50.036 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:20:50.070 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:20:51.147 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:20:51.180 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:20:55.152 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:20:55.182 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:20:56.262 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:20:56.292 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:21:00.264 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:21:00.293 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:21:01.376 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:21:01.403 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:21:05.377 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:21:05.404 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:21:06.488 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:21:06.515 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:21:10.489 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:21:10.516 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:21:11.599 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:21:11.626 DEBUG:teuthology.orchestra.console.smithi139:check power output: 2023-07-28T18:21:15.601 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:21:15.628 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:21:15.739 DEBUG:teuthology.orchestra.console.smithi139:check power output: Chassis Power is off 2023-07-28T18:21:15.840 INFO:teuthology.orchestra.console.smithi139:Power off completed 2023-07-28T18:21:20.945 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:21:25.980 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:21:26.014 INFO:teuthology.lock.ops:unlocked: smithi139.front.sepia.ceph.com 2023-07-28T18:21:27.089 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:21:31.092 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:21:32.202 DEBUG:teuthology.orchestra.console.smithi007:check power output: 2023-07-28T18:21:36.203 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-28T18:21:36.313 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2023-07-28T18:21:36.414 INFO:teuthology.orchestra.console.smithi007:Power off completed 2023-07-28T18:21:42.369 INFO:teuthology.lock.ops:unlocked: smithi007.front.sepia.ceph.com 2023-07-28T18:21:52.604 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/dispatcher/supervisor.py", line 57, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_3117e89c1b049bcc0726b9aaa0facf3451bd8045/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi007's OS to be centos 8 but found rhel 8.4