2024-01-21T21:47:49.635 INFO:root:teuthology version: 0.0.1.dev249+gcd45576
2024-01-21T21:47:49.636 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-21T21:47:49.821 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f4a979a1f40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f4a98a07c10>, signals=[15])
2024-01-21T21:47:49.822 INFO:teuthology.lock.ops:Start node 'smithi093.front.sepia.ceph.com' reimaging
2024-01-21T21:47:49.822 INFO:teuthology.lock.ops:Updating [smithi093.front.sepia.ceph.com]: reset os type and version on server
2024-01-21T21:47:49.822 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server
2024-01-21T21:47:49.852 INFO:teuthology.lock.ops:Start node 'smithi141.front.sepia.ceph.com' reimaging
2024-01-21T21:47:49.852 INFO:teuthology.lock.ops:Updating [smithi141.front.sepia.ceph.com]: reset os type and version on server
2024-01-21T21:47:49.852 INFO:teuthology.lock.ops:Updating smithi141.front.sepia.ceph.com on lock server
2024-01-21T21:47:49.876 INFO:teuthology.lock.ops:Start node 'smithi161.front.sepia.ceph.com' reimaging
2024-01-21T21:47:49.876 INFO:teuthology.lock.ops:Updating [smithi161.front.sepia.ceph.com]: reset os type and version on server
2024-01-21T21:47:49.876 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server
2024-01-21T21:47:49.969 DEBUG:teuthology.provision.fog.smithi093:Requesting image smithi_rhel_8.6 (ID 5)
2024-01-21T21:47:49.993 DEBUG:teuthology.provision.fog.smithi141:Requesting image smithi_rhel_8.6 (ID 5)
2024-01-21T21:47:50.000 DEBUG:teuthology.provision.fog.smithi161:Requesting image smithi_rhel_8.6 (ID 5)
2024-01-21T21:47:50.015 INFO:teuthology.provision.fog.smithi093:Scheduling deploy of rhel 8.6
2024-01-21T21:47:50.032 INFO:teuthology.provision.fog.smithi141:Scheduling deploy of rhel 8.6
2024-01-21T21:47:50.041 INFO:teuthology.provision.fog.smithi161:Scheduling deploy of rhel 8.6
2024-01-21T21:47:50.259 INFO:teuthology.orchestra.console.smithi093:Power off
2024-01-21T21:47:50.259 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-21T21:47:50.274 INFO:teuthology.orchestra.console.smithi161:Power off
2024-01-21T21:47:50.274 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-21T21:47:50.287 INFO:teuthology.orchestra.console.smithi141:Power off
2024-01-21T21:47:50.287 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-21T21:47:50.298 DEBUG:teuthology.orchestra.console.smithi161:power off output: Chassis Power Control: Down/Off
2024-01-21T21:47:50.299 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:47:50.305 DEBUG:teuthology.orchestra.console.smithi093:power off output: Chassis Power Control: Down/Off
2024-01-21T21:47:50.305 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:47:50.312 DEBUG:teuthology.orchestra.console.smithi141:power off output: Chassis Power Control: Down/Off
2024-01-21T21:47:50.313 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:47:50.322 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off
2024-01-21T21:47:50.325 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is off
2024-01-21T21:47:50.336 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is off
2024-01-21T21:47:50.423 INFO:teuthology.orchestra.console.smithi161:Power off completed
2024-01-21T21:47:50.426 INFO:teuthology.orchestra.console.smithi093:Power off completed
2024-01-21T21:47:50.437 INFO:teuthology.orchestra.console.smithi141:Power off completed
2024-01-21T21:47:50.525 INFO:teuthology.orchestra.console.smithi161:Power on
2024-01-21T21:47:50.525 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-21T21:47:50.532 INFO:teuthology.orchestra.console.smithi093:Power on
2024-01-21T21:47:50.533 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-21T21:47:50.539 INFO:teuthology.orchestra.console.smithi141:Power on
2024-01-21T21:47:50.539 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-21T21:47:50.548 DEBUG:teuthology.orchestra.console.smithi161:power on output: Chassis Power Control: Up/On
2024-01-21T21:47:50.548 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:47:50.556 DEBUG:teuthology.orchestra.console.smithi093:power on output: Chassis Power Control: Up/On
2024-01-21T21:47:50.556 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:47:50.563 DEBUG:teuthology.orchestra.console.smithi141:power on output: Chassis Power Control: Up/On
2024-01-21T21:47:50.564 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:47:50.572 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off
2024-01-21T21:47:50.576 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is off
2024-01-21T21:47:50.584 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is off
2024-01-21T21:47:54.577 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:47:54.584 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:47:54.591 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:47:54.687 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on
2024-01-21T21:47:54.693 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is on
2024-01-21T21:47:54.699 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is off
2024-01-21T21:47:54.787 INFO:teuthology.orchestra.console.smithi161:Power on completed
2024-01-21T21:47:54.794 INFO:teuthology.orchestra.console.smithi093:Power on completed
2024-01-21T21:47:54.889 INFO:teuthology.provision.fog.smithi161:Waiting for deploy to finish
2024-01-21T21:47:54.895 INFO:teuthology.provision.fog.smithi093:Waiting for deploy to finish
2024-01-21T21:47:58.702 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:47:58.811 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is on
2024-01-21T21:47:58.912 INFO:teuthology.orchestra.console.smithi141:Power on completed
2024-01-21T21:47:59.012 INFO:teuthology.provision.fog.smithi141:Waiting for deploy to finish
2024-01-21T21:48:55.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:48:58.368 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-21T21:49:04.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:49:07.456 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-21T21:49:13.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:49:16.540 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-21T21:49:22.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:49:25.596 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-21T21:49:31.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:49:34.656 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-21T21:49:40.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:49:43.708 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-21T21:49:49.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:49:52.764 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-21T21:49:58.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:50:01.827 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-21T21:50:07.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:50:10.908 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-21T21:50:16.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:50:27.136 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-21T21:50:29.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:50:33.141 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:50:36.220 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-21T21:50:40.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:50:42.222 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:50:42.357 ERROR:teuthology.orchestra.connection:Error authenticating with smithi161.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-01-21T21:50:46.491 ERROR:teuthology.orchestra.connection:Error authenticating with smithi161.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-01-21T21:50:48.380 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141
2024-01-21T21:50:53.582 ERROR:teuthology.orchestra.connection:Error authenticating with smithi161.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-01-21T21:50:54.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:50:57.436 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141
2024-01-21T21:50:59.388 WARNING:teuthology.provision.fog.smithi093:[Errno None] Unable to connect to port 22 on 172.21.15.93
2024-01-21T21:51:03.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:51:03.679 ERROR:teuthology.orchestra.connection:Error authenticating with smithi161.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-01-21T21:51:05.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:51:06.496 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141
2024-01-21T21:51:08.444 WARNING:teuthology.provision.fog.smithi093:[Errno None] Unable to connect to port 22 on 172.21.15.93
2024-01-21T21:51:12.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:51:14.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:51:15.552 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141
2024-01-21T21:51:16.812 ERROR:teuthology.orchestra.connection:Error authenticating with smithi161.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-01-21T21:51:17.500 WARNING:teuthology.provision.fog.smithi093:[Errno None] Unable to connect to port 22 on 172.21.15.93
2024-01-21T21:51:21.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:51:23.502 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:51:24.607 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141
2024-01-21T21:51:26.556 WARNING:teuthology.provision.fog.smithi093:[Errno None] Unable to connect to port 22 on 172.21.15.93
2024-01-21T21:51:30.608 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:51:32.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:51:32.943 ERROR:teuthology.orchestra.connection:Error authenticating with smithi161.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-01-21T21:51:51.100 WARNING:teuthology.provision.fog.smithi093:[Errno None] Unable to connect to port 22 on 172.21.15.93
2024-01-21T21:51:57.101 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:52:00.160 WARNING:teuthology.provision.fog.smithi093:[Errno None] Unable to connect to port 22 on 172.21.15.93
2024-01-21T21:52:06.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:52:09.216 WARNING:teuthology.provision.fog.smithi093:[Errno None] Unable to connect to port 22 on 172.21.15.93
2024-01-21T21:52:15.217 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:52:16.037 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-21T21:52:16.037 DEBUG:teuthology.orchestra.run.smithi093:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-21T21:52:30.610 WARNING:teuthology.provision.fog.smithi141:timed out
2024-01-21T21:52:31.534 INFO:teuthology.provision.fog.smithi093:Node is ready
2024-01-21T21:52:31.535 DEBUG:teuthology.orchestra.run.smithi093:> hostname
2024-01-21T21:52:31.554 INFO:teuthology.orchestra.run.smithi093.stdout:smithi093.front.sepia.ceph.com
2024-01-21T21:52:31.555 DEBUG:teuthology.orchestra.run.smithi093:> grep smithi093.front.sepia.ceph.com /etc/hosts
2024-01-21T21:52:31.613 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93 smithi093.front.sepia.ceph.com smithi093
2024-01-21T21:52:31.614 DEBUG:teuthology.orchestra.run.smithi093:> sudo hostname smithi093
2024-01-21T21:52:31.731 DEBUG:teuthology.orchestra.run.smithi093:> sudo sed -i -e 's/smithi093.front.sepia.ceph.com/smithi093/g' /etc/hosts
2024-01-21T21:52:31.759 DEBUG:teuthology.orchestra.run.smithi093:> sudo sed -i -e 's/172.21.15.93/172.21.15.93/g' /etc/hosts
2024-01-21T21:52:31.826 DEBUG:teuthology.orchestra.run.smithi093:> sudo sed -i -e 's/smithi093.front.sepia.ceph.com/smithi093/g' /etc/hostname
2024-01-21T21:52:31.893 DEBUG:teuthology.orchestra.run.smithi093:> sudo hostname smithi093
2024-01-21T21:52:31.967 DEBUG:teuthology.orchestra.run.smithi093:> cat /etc/os-release
2024-01-21T21:52:32.027 INFO:teuthology.orchestra.run.smithi093.stdout:NAME="Red Hat Enterprise Linux"
2024-01-21T21:52:32.027 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION="8.6 (Ootpa)"
2024-01-21T21:52:32.027 INFO:teuthology.orchestra.run.smithi093.stdout:ID="rhel"
2024-01-21T21:52:32.028 INFO:teuthology.orchestra.run.smithi093.stdout:ID_LIKE="fedora"
2024-01-21T21:52:32.028 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_ID="8.6"
2024-01-21T21:52:32.028 INFO:teuthology.orchestra.run.smithi093.stdout:PLATFORM_ID="platform:el8"
2024-01-21T21:52:32.028 INFO:teuthology.orchestra.run.smithi093.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2024-01-21T21:52:32.028 INFO:teuthology.orchestra.run.smithi093.stdout:ANSI_COLOR="0;31"
2024-01-21T21:52:32.028 INFO:teuthology.orchestra.run.smithi093.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2024-01-21T21:52:32.028 INFO:teuthology.orchestra.run.smithi093.stdout:HOME_URL="https://www.redhat.com/"
2024-01-21T21:52:32.028 INFO:teuthology.orchestra.run.smithi093.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2024-01-21T21:52:32.028 INFO:teuthology.orchestra.run.smithi093.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-21T21:52:32.028 INFO:teuthology.orchestra.run.smithi093.stdout:
2024-01-21T21:52:32.028 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2024-01-21T21:52:32.028 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2024-01-21T21:52:32.028 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2024-01-21T21:52:32.028 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2024-01-21T21:52:32.029 INFO:teuthology.provision.fog.smithi093:Deploy complete!
2024-01-21T21:52:32.030 DEBUG:teuthology.parallel:result is None
2024-01-21T21:52:36.616 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:52:36.716 ERROR:paramiko.transport:Unknown exception: The socket has already been closed by another greenlet
2024-01-21T21:52:36.773 ERROR:paramiko.transport:Traceback (most recent call last):
2024-01-21T21:52:36.773 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2159, in run
2024-01-21T21:52:36.773 ERROR:paramiko.transport:    ptype, m = self.packetizer.read_message()
2024-01-21T21:52:36.773 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 463, in read_message
2024-01-21T21:52:36.773 ERROR:paramiko.transport:    header = self.read_all(self.__block_size_in, check_rekey=True)
2024-01-21T21:52:36.773 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 308, in read_all
2024-01-21T21:52:36.773 ERROR:paramiko.transport:    x = self.__socket.recv(n)
2024-01-21T21:52:36.773 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/gevent/_socketcommon.py", line 666, in recv
2024-01-21T21:52:36.773 ERROR:paramiko.transport:    self._wait(self._read_event)
2024-01-21T21:52:36.773 ERROR:paramiko.transport:  File "src/gevent/_hub_primitives.py", line 317, in gevent._gevent_c_hub_primitives.wait_on_socket
2024-01-21T21:52:36.773 ERROR:paramiko.transport:  File "src/gevent/_hub_primitives.py", line 321, in gevent._gevent_c_hub_primitives.wait_on_socket
2024-01-21T21:52:36.774 ERROR:paramiko.transport:gevent.exceptions.ConcurrentObjectUseError: The socket has already been closed by another greenlet
2024-01-21T21:52:36.774 ERROR:paramiko.transport:
2024-01-21T21:52:36.976 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-21T21:52:36.976 DEBUG:teuthology.orchestra.run.smithi141:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-21T21:52:37.198 INFO:teuthology.provision.fog.smithi141:Node is ready
2024-01-21T21:52:37.198 DEBUG:teuthology.orchestra.run.smithi141:> hostname
2024-01-21T21:52:37.216 INFO:teuthology.orchestra.run.smithi141.stdout:smithi141
2024-01-21T21:52:37.216 DEBUG:teuthology.orchestra.run.smithi141:> grep smithi141 /etc/hosts
2024-01-21T21:52:37.274 INFO:teuthology.orchestra.run.smithi141.stdout:172.21.15.141 smithi141.front.sepia.ceph.com smithi141
2024-01-21T21:52:37.275 DEBUG:teuthology.orchestra.run.smithi141:> sudo hostname smithi141
2024-01-21T21:52:37.378 DEBUG:teuthology.orchestra.run.smithi141:> sudo sed -i -e 's/smithi141/smithi141/g' /etc/hosts
2024-01-21T21:52:37.407 DEBUG:teuthology.orchestra.run.smithi141:> sudo sed -i -e 's/172.21.15.141/172.21.15.141/g' /etc/hosts
2024-01-21T21:52:37.474 DEBUG:teuthology.orchestra.run.smithi141:> sudo sed -i -e 's/smithi141/smithi141/g' /etc/hostname
2024-01-21T21:52:37.542 DEBUG:teuthology.orchestra.run.smithi141:> sudo hostname smithi141
2024-01-21T21:52:37.609 DEBUG:teuthology.orchestra.run.smithi141:> cat /etc/os-release
2024-01-21T21:52:37.666 INFO:teuthology.orchestra.run.smithi141.stdout:NAME="Red Hat Enterprise Linux"
2024-01-21T21:52:37.666 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION="8.6 (Ootpa)"
2024-01-21T21:52:37.666 INFO:teuthology.orchestra.run.smithi141.stdout:ID="rhel"
2024-01-21T21:52:37.666 INFO:teuthology.orchestra.run.smithi141.stdout:ID_LIKE="fedora"
2024-01-21T21:52:37.666 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION_ID="8.6"
2024-01-21T21:52:37.666 INFO:teuthology.orchestra.run.smithi141.stdout:PLATFORM_ID="platform:el8"
2024-01-21T21:52:37.666 INFO:teuthology.orchestra.run.smithi141.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2024-01-21T21:52:37.666 INFO:teuthology.orchestra.run.smithi141.stdout:ANSI_COLOR="0;31"
2024-01-21T21:52:37.666 INFO:teuthology.orchestra.run.smithi141.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2024-01-21T21:52:37.666 INFO:teuthology.orchestra.run.smithi141.stdout:HOME_URL="https://www.redhat.com/"
2024-01-21T21:52:37.666 INFO:teuthology.orchestra.run.smithi141.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2024-01-21T21:52:37.666 INFO:teuthology.orchestra.run.smithi141.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-21T21:52:37.667 INFO:teuthology.orchestra.run.smithi141.stdout:
2024-01-21T21:52:37.667 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2024-01-21T21:52:37.667 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2024-01-21T21:52:37.667 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2024-01-21T21:52:37.667 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2024-01-21T21:52:37.667 INFO:teuthology.provision.fog.smithi141:Deploy complete!
2024-01-21T21:52:37.668 DEBUG:teuthology.parallel:result is None
2024-01-21T21:52:51.946 WARNING:teuthology.provision.fog.smithi161:timed out
2024-01-21T21:52:57.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:53:00.000 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-21T21:53:06.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:53:09.056 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-21T21:53:15.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:53:15.196 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-21T21:53:21.197 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:53:21.212 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-21T21:53:27.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:53:30.268 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-21T21:53:36.269 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:53:39.324 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-21T21:53:45.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:54:45.330 WARNING:teuthology.provision.fog.smithi161:timed out
2024-01-21T21:54:51.332 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-21T21:54:51.586 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-21T21:54:51.586 DEBUG:teuthology.orchestra.run.smithi161:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-21T21:54:51.823 INFO:teuthology.provision.fog.smithi161:Node is ready
2024-01-21T21:54:51.823 DEBUG:teuthology.orchestra.run.smithi161:> hostname
2024-01-21T21:54:51.883 INFO:teuthology.orchestra.run.smithi161.stdout:smithi161.front.sepia.ceph.com
2024-01-21T21:54:51.883 DEBUG:teuthology.orchestra.run.smithi161:> grep smithi161.front.sepia.ceph.com /etc/hosts
2024-01-21T21:54:51.942 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161 smithi161.front.sepia.ceph.com smithi161
2024-01-21T21:54:51.943 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname smithi161
2024-01-21T21:54:52.052 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/smithi161.front.sepia.ceph.com/smithi161/g' /etc/hosts
2024-01-21T21:54:52.082 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/172.21.15.161/172.21.15.161/g' /etc/hosts
2024-01-21T21:54:52.151 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/smithi161.front.sepia.ceph.com/smithi161/g' /etc/hostname
2024-01-21T21:54:52.233 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname smithi161
2024-01-21T21:54:52.299 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release
2024-01-21T21:54:52.356 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream"
2024-01-21T21:54:52.356 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8"
2024-01-21T21:54:52.356 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos"
2024-01-21T21:54:52.356 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora"
2024-01-21T21:54:52.356 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8"
2024-01-21T21:54:52.356 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8"
2024-01-21T21:54:52.356 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 8"
2024-01-21T21:54:52.356 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31"
2024-01-21T21:54:52.357 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2024-01-21T21:54:52.357 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/"
2024-01-21T21:54:52.357 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-21T21:54:52.357 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2024-01-21T21:54:52.357 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-21T21:54:52.358 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/provision/fog.py", line 92, in create
    self._verify_installed_os()
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/provision/fog.py", line 347, in _verify_installed_os
    raise RuntimeError(
RuntimeError: Expected smithi161's OS to be rhel 8.6 but found centos 8
2024-01-21T21:54:52.360 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-21T21:54:52.651 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=7e6b47864dea4b42af8df0d67feef0a8
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/provision/fog.py", line 92, in create
    self._verify_installed_os()
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/provision/fog.py", line 347, in _verify_installed_os
    raise RuntimeError(
RuntimeError: Expected smithi161's OS to be rhel 8.6 but found centos 8
2024-01-21T21:54:52.653 INFO:teuthology.nuke:Checking targets against current locks
2024-01-21T21:54:52.723 DEBUG:teuthology.nuke:shortname: smithi093
2024-01-21T21:54:52.723 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-21T21:54:52.745 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi093.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522001', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-21 21:47:48.559012', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLmQMtWuiBs38Vai/2FnEOQRV5Eb2vsK0mlbDRHjmKHuGkO/UfOljuvK9r3Q67SlWlGu4nwLaOmdNzeG0ABHNzw='}
2024-01-21T21:54:52.748 DEBUG:teuthology.nuke:shortname: smithi141
2024-01-21T21:54:52.748 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-21T21:54:52.765 DEBUG:teuthology.nuke:shortname: smithi161
2024-01-21T21:54:52.765 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-21T21:54:52.770 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi141.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522001', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-21 21:47:48.557555', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCDGMlcb2s9uxA75A8n5glPBoIvrADEyHphXv+AXyG150rYCMj5RNlAObs0e/yY3oxC4RdOLAuYTRxFgfNHhQKs='}
2024-01-21T21:54:52.795 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522007', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-21 21:48:51.347982', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIxI0lEjvdII/L5+2CVVqBXKDP8ZxIRLpaC4SGCPmVIEBdrZAE2phdPQ4Z6/trGHFtLBUyX75QbhugmpGyx/hMA='}
2024-01-21T21:54:52.801 INFO:teuthology.orchestra.console.smithi093:Power off
2024-01-21T21:54:52.801 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-21T21:54:52.824 DEBUG:teuthology.orchestra.console.smithi093:power off output: Chassis Power Control: Down/Off
2024-01-21T21:54:52.824 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:54:52.840 INFO:teuthology.orchestra.console.smithi141:Power off
2024-01-21T21:54:52.841 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-21T21:54:52.854 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is on
2024-01-21T21:54:52.862 DEBUG:teuthology.orchestra.console.smithi141:power off output: Chassis Power Control: Down/Off
2024-01-21T21:54:52.862 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:54:52.881 INFO:teuthology.orchestra.console.smithi161:Power off
2024-01-21T21:54:52.882 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-21T21:54:52.891 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is on
2024-01-21T21:54:52.905 DEBUG:teuthology.orchestra.console.smithi161:power off output: Chassis Power Control: Down/Off
2024-01-21T21:54:52.905 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:54:52.929 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on
2024-01-21T21:54:56.854 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:54:56.892 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:54:56.931 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:54:56.965 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is on
2024-01-21T21:54:57.002 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is on
2024-01-21T21:54:57.041 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on
2024-01-21T21:55:00.966 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:55:01.002 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:55:01.043 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:55:01.080 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is on
2024-01-21T21:55:01.113 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is on
2024-01-21T21:55:01.153 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on
2024-01-21T21:55:05.082 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:55:05.113 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:55:05.154 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-21T21:55:05.192 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is off
2024-01-21T21:55:05.223 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is off
2024-01-21T21:55:05.264 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off
2024-01-21T21:55:05.293 INFO:teuthology.orchestra.console.smithi093:Power off completed
2024-01-21T21:55:05.324 INFO:teuthology.orchestra.console.smithi141:Power off completed
2024-01-21T21:55:05.364 INFO:teuthology.orchestra.console.smithi161:Power off completed
2024-01-21T21:55:05.445 INFO:teuthology.lock.ops:unlocked: smithi093.front.sepia.ceph.com
2024-01-21T21:55:05.459 INFO:teuthology.lock.ops:unlocked: smithi141.front.sepia.ceph.com
2024-01-21T21:55:05.510 INFO:teuthology.lock.ops:unlocked: smithi161.front.sepia.ceph.com
2024-01-21T21:55:05.828 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/provision/fog.py", line 92, in create
    self._verify_installed_os()
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/provision/fog.py", line 347, in _verify_installed_os
    raise RuntimeError(
RuntimeError: Expected smithi161's OS to be rhel 8.6 but found centos 8