2023-12-11T04:36:13.191 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e
2023-12-11T04:36:13.192 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T04:36:13.549 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f138e2cd2b0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f138f224940>, signals=[15])
2023-12-11T04:36:13.549 INFO:teuthology.lock.ops:Start node 'smithi066.front.sepia.ceph.com' reimaging
2023-12-11T04:36:13.549 INFO:teuthology.lock.ops:Updating [smithi066.front.sepia.ceph.com]: reset os type and version on server
2023-12-11T04:36:13.550 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server
2023-12-11T04:36:13.574 INFO:teuthology.lock.ops:Start node 'smithi203.front.sepia.ceph.com' reimaging
2023-12-11T04:36:13.574 INFO:teuthology.lock.ops:Updating [smithi203.front.sepia.ceph.com]: reset os type and version on server
2023-12-11T04:36:13.574 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server
2023-12-11T04:36:13.695 DEBUG:teuthology.provision.fog.smithi066:Requesting image smithi_centos_8.stream (ID 3)
2023-12-11T04:36:13.721 DEBUG:teuthology.provision.fog.smithi203:Requesting image smithi_centos_8.stream (ID 3)
2023-12-11T04:36:13.744 INFO:teuthology.provision.fog.smithi066:Scheduling deploy of centos 8.stream
2023-12-11T04:36:13.772 INFO:teuthology.provision.fog.smithi203:Scheduling deploy of centos 8.stream
2023-12-11T04:36:13.826 DEBUG:teuthology.provision.fog.smithi203:Canceling deploy task with ID 659439
2023-12-11T04:36:14.018 INFO:teuthology.orchestra.console.smithi066:Power off
2023-12-11T04:36:14.018 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-11T04:36:14.113 INFO:teuthology.orchestra.console.smithi203:Power off
2023-12-11T04:36:14.113 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-11T04:36:14.127 DEBUG:teuthology.orchestra.console.smithi066:power off output: Chassis Power Control: Down/Off
2023-12-11T04:36:14.128 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T04:36:14.193 DEBUG:teuthology.orchestra.console.smithi203:power off output: Chassis Power Control: Down/Off
2023-12-11T04:36:14.193 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T04:36:14.203 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is off
2023-12-11T04:36:14.282 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is off
2023-12-11T04:36:14.304 INFO:teuthology.orchestra.console.smithi066:Power off completed
2023-12-11T04:36:14.384 INFO:teuthology.orchestra.console.smithi203:Power off completed
2023-12-11T04:36:14.407 INFO:teuthology.orchestra.console.smithi066:Power on
2023-12-11T04:36:14.407 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-11T04:36:14.484 INFO:teuthology.orchestra.console.smithi203:Power on
2023-12-11T04:36:14.485 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-11T04:36:14.508 DEBUG:teuthology.orchestra.console.smithi066:power on output: Chassis Power Control: Up/On
2023-12-11T04:36:14.509 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T04:36:14.607 DEBUG:teuthology.orchestra.console.smithi203:power on output: Chassis Power Control: Up/On
2023-12-11T04:36:14.607 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T04:36:14.626 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is off
2023-12-11T04:36:14.723 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is off
2023-12-11T04:36:18.633 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T04:36:18.727 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T04:36:18.760 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is off
2023-12-11T04:36:18.862 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on
2023-12-11T04:36:18.963 INFO:teuthology.orchestra.console.smithi203:Power on completed
2023-12-11T04:36:19.063 INFO:teuthology.provision.fog.smithi203:Waiting for deploy to finish
2023-12-11T04:36:22.763 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T04:36:22.905 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on
2023-12-11T04:36:23.005 INFO:teuthology.orchestra.console.smithi066:Power on completed
2023-12-11T04:36:23.106 INFO:teuthology.provision.fog.smithi066:Waiting for deploy to finish
2023-12-11T04:39:04.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:39:53.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:39:57.031 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T04:40:03.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:40:04.817 WARNING:teuthology.provision.fog.smithi203:timed out
2023-12-11T04:40:06.114 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T04:40:10.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:40:12.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:40:15.170 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T04:40:21.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:40:24.260 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T04:40:30.266 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:40:33.350 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T04:40:39.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:40:42.434 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T04:40:48.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:40:51.491 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T04:40:57.495 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:41:00.549 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T04:41:06.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:41:09.637 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T04:41:10.827 WARNING:teuthology.provision.fog.smithi203:timed out
2023-12-11T04:41:15.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:41:16.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:41:16.836 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2023-12-11T04:41:18.725 WARNING:teuthology.provision.fog.smithi066:[Errno None] Unable to connect to port 22 on 172.21.15.66
2023-12-11T04:41:22.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:41:24.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:42:22.901 WARNING:teuthology.provision.fog.smithi203:timed out
2023-12-11T04:42:24.730 WARNING:teuthology.provision.fog.smithi066:timed out
2023-12-11T04:42:28.905 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:42:30.733 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:42:31.049 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-11T04:42:31.050 DEBUG:teuthology.orchestra.run.smithi066:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-11T04:42:31.287 INFO:teuthology.provision.fog.smithi066:Node is ready
2023-12-11T04:42:31.287 DEBUG:teuthology.orchestra.run.smithi066:> hostname
2023-12-11T04:42:31.308 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066.front.sepia.ceph.com
2023-12-11T04:42:31.309 DEBUG:teuthology.orchestra.run.smithi066:> grep smithi066.front.sepia.ceph.com /etc/hosts
2023-12-11T04:42:31.368 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66 smithi066.front.sepia.ceph.com smithi066
2023-12-11T04:42:31.369 DEBUG:teuthology.orchestra.run.smithi066:> sudo hostname smithi066
2023-12-11T04:42:31.465 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i -e 's/smithi066.front.sepia.ceph.com/smithi066/g' /etc/hosts
2023-12-11T04:42:31.495 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i -e 's/172.21.15.66/172.21.15.66/g' /etc/hosts
2023-12-11T04:42:31.563 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i -e 's/smithi066.front.sepia.ceph.com/smithi066/g' /etc/hostname
2023-12-11T04:42:31.633 DEBUG:teuthology.orchestra.run.smithi066:> sudo hostname smithi066
2023-12-11T04:42:31.705 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release
2023-12-11T04:42:31.764 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="CentOS Stream"
2023-12-11T04:42:31.764 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="8"
2023-12-11T04:42:31.764 INFO:teuthology.orchestra.run.smithi066.stdout:ID="centos"
2023-12-11T04:42:31.764 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="rhel fedora"
2023-12-11T04:42:31.764 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="8"
2023-12-11T04:42:31.764 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el8"
2023-12-11T04:42:31.764 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="CentOS Stream 8"
2023-12-11T04:42:31.765 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31"
2023-12-11T04:42:31.765 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-12-11T04:42:31.765 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://centos.org/"
2023-12-11T04:42:31.765 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-11T04:42:31.765 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-11T04:42:31.765 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-12-11T04:42:31.766 INFO:teuthology.provision.fog.smithi066:Deploy complete!
2023-12-11T04:42:31.767 DEBUG:teuthology.parallel:result is None
2023-12-11T04:42:31.970 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2023-12-11T04:42:37.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:42:41.035 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2023-12-11T04:42:47.042 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:43:18.628 ERROR:teuthology.orchestra.connection:Error authenticating with smithi203.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-12-11T04:43:22.731 ERROR:teuthology.orchestra.connection:Error authenticating with smithi203.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-12-11T04:43:29.827 ERROR:teuthology.orchestra.connection:Error authenticating with smithi203.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-12-11T04:43:39.953 ERROR:teuthology.orchestra.connection:Error authenticating with smithi203.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-12-11T04:43:53.083 ERROR:teuthology.orchestra.connection:Error authenticating with smithi203.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-12-11T04:44:09.214 ERROR:teuthology.orchestra.connection:Error authenticating with smithi203.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-12-11T04:45:02.690 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2023-12-11T04:45:08.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:45:11.746 WARNING:teuthology.provision.fog.smithi203:[Errno None] Unable to connect to port 22 on 172.21.15.203
2023-12-11T04:45:17.748 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:46:17.751 WARNING:teuthology.provision.fog.smithi203:timed out
2023-12-11T04:46:23.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-11T04:46:24.077 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-11T04:46:24.077 DEBUG:teuthology.orchestra.run.smithi203:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-11T04:46:24.753 INFO:teuthology.provision.fog.smithi203:Node is ready
2023-12-11T04:46:24.754 DEBUG:teuthology.orchestra.run.smithi203:> hostname
2023-12-11T04:46:24.772 INFO:teuthology.orchestra.run.smithi203.stdout:smithi203.front.sepia.ceph.com
2023-12-11T04:46:24.773 DEBUG:teuthology.orchestra.run.smithi203:> grep smithi203.front.sepia.ceph.com /etc/hosts
2023-12-11T04:46:24.831 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.15.203 smithi203.front.sepia.ceph.com smithi203
2023-12-11T04:46:24.832 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname smithi203
2023-12-11T04:46:24.951 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i -e 's/smithi203.front.sepia.ceph.com/smithi203/g' /etc/hosts
2023-12-11T04:46:24.981 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i -e 's/172.21.15.203/172.21.15.203/g' /etc/hosts
2023-12-11T04:46:25.051 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i -e 's/smithi203.front.sepia.ceph.com/smithi203/g' /etc/hostname
2023-12-11T04:46:25.119 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname smithi203
2023-12-11T04:46:25.187 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release
2023-12-11T04:46:25.244 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="Red Hat Enterprise Linux"
2023-12-11T04:46:25.244 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="8.6 (Ootpa)"
2023-12-11T04:46:25.244 INFO:teuthology.orchestra.run.smithi203.stdout:ID="rhel"
2023-12-11T04:46:25.244 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="fedora"
2023-12-11T04:46:25.244 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="8.6"
2023-12-11T04:46:25.245 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el8"
2023-12-11T04:46:25.245 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-12-11T04:46:25.245 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31"
2023-12-11T04:46:25.245 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-12-11T04:46:25.245 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://www.redhat.com/"
2023-12-11T04:46:25.245 INFO:teuthology.orchestra.run.smithi203.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-12-11T04:46:25.245 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-11T04:46:25.245 INFO:teuthology.orchestra.run.smithi203.stdout:
2023-12-11T04:46:25.245 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-11T04:46:25.245 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-12-11T04:46:25.245 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-12-11T04:46:25.245 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-12-11T04:46:25.247 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/provision/fog.py", line 92, in create
    self._verify_installed_os()
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/provision/fog.py", line 347, in _verify_installed_os
    raise RuntimeError(
RuntimeError: Expected smithi203's OS to be centos 8 but found rhel 8.6
2023-12-11T04:46:25.249 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-11T04:46:25.399 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=19fdc51b723e4faabd83b83674b1f69e
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/provision/fog.py", line 92, in create
    self._verify_installed_os()
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/provision/fog.py", line 347, in _verify_installed_os
    raise RuntimeError(
RuntimeError: Expected smithi203's OS to be centos 8 but found rhel 8.6
2023-12-11T04:46:25.402 INFO:teuthology.nuke:Checking targets against current locks
2023-12-11T04:46:25.471 DEBUG:teuthology.nuke:shortname: smithi066
2023-12-11T04:46:25.471 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-11T04:46:25.493 DEBUG:teuthology.nuke:shortname: smithi203
2023-12-11T04:46:25.493 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-11T04:46:25.497 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2023-12-11_04:19:36-orch:cephadm-wip-adk-testing-2023-12-10-1329-distro-default-smithi/7486613', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-11 04:36:12.167916', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMbwxfMUu+DFTZXigtgYJVyoBl9xXfJG9Mw+yTD3/Kkf6ENhOgaPxiOpTNqXUgCiu7ql5GPJRoEwIvowu0CyJQM='}
2023-12-11T04:46:25.518 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2023-12-11_04:19:36-orch:cephadm-wip-adk-testing-2023-12-10-1329-distro-default-smithi/7486614', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-11 04:41:23.303796', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC7BOvm5OnNyLJV79ROOICjd+yBMTYjM0LzOEi0X4QrnAqbsska3ttyiCnaX0pBEUBJbmUKOPFln1nBzsxfSRzA='}
2023-12-11T04:46:25.547 INFO:teuthology.orchestra.console.smithi066:Power off
2023-12-11T04:46:25.547 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-11T04:46:25.572 DEBUG:teuthology.orchestra.console.smithi066:power off output: Chassis Power Control: Down/Off
2023-12-11T04:46:25.573 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T04:46:25.590 INFO:teuthology.orchestra.console.smithi203:Power off
2023-12-11T04:46:25.591 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-11T04:46:25.602 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on
2023-12-11T04:46:25.616 DEBUG:teuthology.orchestra.console.smithi203:power off output: Chassis Power Control: Down/Off
2023-12-11T04:46:25.617 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T04:46:25.642 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on
2023-12-11T04:46:29.603 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T04:46:29.643 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T04:46:29.713 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on
2023-12-11T04:46:29.754 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on
2023-12-11T04:46:33.714 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T04:46:33.755 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T04:46:33.831 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is on
2023-12-11T04:46:33.866 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is off
2023-12-11T04:46:33.974 INFO:teuthology.orchestra.console.smithi203:Power off completed
2023-12-11T04:46:34.131 INFO:teuthology.lock.ops:unlocked: smithi203.front.sepia.ceph.com
2023-12-11T04:46:37.832 DEBUG:teuthology.orchestra.console.smithi066:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-11T04:46:37.946 DEBUG:teuthology.orchestra.console.smithi066:check power output: Chassis Power is off
2023-12-11T04:46:38.048 INFO:teuthology.orchestra.console.smithi066:Power off completed
2023-12-11T04:46:38.228 INFO:teuthology.lock.ops:unlocked: smithi066.front.sepia.ceph.com
2023-12-11T04:46:38.514 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/provision/fog.py", line 92, in create
    self._verify_installed_os()
  File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/provision/fog.py", line 347, in _verify_installed_os
    raise RuntimeError(
RuntimeError: Expected smithi203's OS to be centos 8 but found rhel 8.6