2024-03-07T19:24:10.491 INFO:root:teuthology version: 0.0.1.dev262+ge691533
2024-03-07T19:24:10.492 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T19:24:10.723 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f1ac6057f70>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f1ac70b8c10>, signals=[15])
2024-03-07T19:24:10.723 INFO:teuthology.lock.ops:Start node 'smithi040.front.sepia.ceph.com' reimaging
2024-03-07T19:24:10.723 INFO:teuthology.lock.ops:Updating [smithi040.front.sepia.ceph.com]: reset os type and version on server
2024-03-07T19:24:10.723 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server
2024-03-07T19:24:10.743 INFO:teuthology.lock.ops:Start node 'smithi098.front.sepia.ceph.com' reimaging
2024-03-07T19:24:10.743 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: reset os type and version on server
2024-03-07T19:24:10.743 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server
2024-03-07T19:24:10.866 DEBUG:teuthology.provision.fog.smithi040:Requesting image smithi_centos_9.stream (ID 10)
2024-03-07T19:24:10.871 DEBUG:teuthology.provision.fog.smithi098:Requesting image smithi_centos_9.stream (ID 10)
2024-03-07T19:24:10.912 INFO:teuthology.provision.fog.smithi098:Scheduling deploy of centos 9.stream
2024-03-07T19:24:10.921 INFO:teuthology.provision.fog.smithi040:Scheduling deploy of centos 9.stream
2024-03-07T19:24:11.212 INFO:teuthology.orchestra.console.smithi098:Power off
2024-03-07T19:24:11.212 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-07T19:24:11.233 INFO:teuthology.orchestra.console.smithi040:Power off
2024-03-07T19:24:11.233 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-07T19:24:11.245 DEBUG:teuthology.orchestra.console.smithi098:power off output: Chassis Power Control: Down/Off
2024-03-07T19:24:11.245 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T19:24:11.265 DEBUG:teuthology.orchestra.console.smithi040:power off output: Chassis Power Control: Down/Off
2024-03-07T19:24:11.265 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T19:24:11.274 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off
2024-03-07T19:24:11.296 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on
2024-03-07T19:24:11.375 INFO:teuthology.orchestra.console.smithi098:Power off completed
2024-03-07T19:24:11.477 INFO:teuthology.orchestra.console.smithi098:Power on
2024-03-07T19:24:11.477 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-07T19:24:11.500 DEBUG:teuthology.orchestra.console.smithi098:power on output: Chassis Power Control: Up/On
2024-03-07T19:24:11.500 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T19:24:11.523 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off
2024-03-07T19:24:15.298 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T19:24:15.412 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on
2024-03-07T19:24:15.524 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T19:24:15.635 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off
2024-03-07T19:24:19.414 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T19:24:19.524 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off
2024-03-07T19:24:19.625 INFO:teuthology.orchestra.console.smithi040:Power off completed
2024-03-07T19:24:19.636 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T19:24:19.725 INFO:teuthology.orchestra.console.smithi040:Power on
2024-03-07T19:24:19.726 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-07T19:24:19.745 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on
2024-03-07T19:24:19.754 DEBUG:teuthology.orchestra.console.smithi040:power on output: Chassis Power Control: Up/On
2024-03-07T19:24:19.755 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T19:24:19.782 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off
2024-03-07T19:24:19.846 INFO:teuthology.orchestra.console.smithi098:Power on completed
2024-03-07T19:24:19.947 INFO:teuthology.provision.fog.smithi098:Waiting for deploy to finish
2024-03-07T19:24:23.786 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T19:24:23.895 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off
2024-03-07T19:24:27.898 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T19:24:28.010 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on
2024-03-07T19:24:28.110 INFO:teuthology.orchestra.console.smithi040:Power on completed
2024-03-07T19:24:28.211 INFO:teuthology.provision.fog.smithi040:Waiting for deploy to finish
2024-03-07T19:24:28.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:24:31.390 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40
2024-03-07T19:24:37.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:24:40.450 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40
2024-03-07T19:24:46.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:24:49.534 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40
2024-03-07T19:24:55.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:24:58.590 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40
2024-03-07T19:25:04.595 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:25:07.646 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40
2024-03-07T19:25:13.653 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:25:16.734 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40
2024-03-07T19:25:22.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:26:22.740 WARNING:teuthology.provision.fog.smithi040:timed out
2024-03-07T19:26:28.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:26:28.882 ERROR:teuthology.orchestra.connection:Error authenticating with smithi040.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-07T19:26:33.016 ERROR:teuthology.orchestra.connection:Error authenticating with smithi040.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-07T19:26:40.148 ERROR:teuthology.orchestra.connection:Error authenticating with smithi040.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-07T19:26:50.283 ERROR:teuthology.orchestra.connection:Error authenticating with smithi040.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-07T19:27:03.417 ERROR:teuthology.orchestra.connection:Error authenticating with smithi040.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-07T19:27:06.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:27:09.566 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2024-03-07T19:27:15.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:27:18.622 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2024-03-07T19:27:19.510 ERROR:teuthology.orchestra.connection:Error authenticating with smithi040.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-07T19:27:24.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:27:27.678 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2024-03-07T19:27:33.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:27:36.734 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2024-03-07T19:27:38.607 ERROR:teuthology.orchestra.connection:Error authenticating with smithi040.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-07T19:27:42.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:27:45.790 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2024-03-07T19:27:51.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:27:54.846 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2024-03-07T19:28:00.738 ERROR:teuthology.orchestra.connection:Error authenticating with smithi040.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-07T19:28:00.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:28:25.873 ERROR:teuthology.orchestra.connection:Error authenticating with smithi040.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-07T19:29:00.850 WARNING:teuthology.provision.fog.smithi098:timed out
2024-03-07T19:29:06.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:29:07.136 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-07T19:29:07.137 DEBUG:teuthology.orchestra.run.smithi098:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-07T19:29:07.448 INFO:teuthology.provision.fog.smithi098:Node is ready
2024-03-07T19:29:07.448 DEBUG:teuthology.orchestra.run.smithi098:> hostname
2024-03-07T19:29:07.463 INFO:teuthology.orchestra.run.smithi098.stdout:smithi098.front.sepia.ceph.com
2024-03-07T19:29:07.463 DEBUG:teuthology.orchestra.run.smithi098:> grep smithi098.front.sepia.ceph.com /etc/hosts
2024-03-07T19:29:07.519 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98 smithi098.front.sepia.ceph.com smithi098
2024-03-07T19:29:07.519 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname smithi098
2024-03-07T19:29:07.632 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/smithi098.front.sepia.ceph.com/smithi098/g' /etc/hosts
2024-03-07T19:29:07.660 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/172.21.15.98/172.21.15.98/g' /etc/hosts
2024-03-07T19:29:07.728 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/smithi098.front.sepia.ceph.com/smithi098/g' /etc/hostname
2024-03-07T19:29:07.794 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname smithi098
2024-03-07T19:29:07.861 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release
2024-03-07T19:29:07.915 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream"
2024-03-07T19:29:07.915 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="9"
2024-03-07T19:29:07.915 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos"
2024-03-07T19:29:07.915 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora"
2024-03-07T19:29:07.915 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="9"
2024-03-07T19:29:07.915 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el9"
2024-03-07T19:29:07.915 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-07T19:29:07.915 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31"
2024-03-07T19:29:07.915 INFO:teuthology.orchestra.run.smithi098.stdout:LOGO="fedora-logo-icon"
2024-03-07T19:29:07.915 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-07T19:29:07.915 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/"
2024-03-07T19:29:07.916 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-07T19:29:07.916 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-07T19:29:07.916 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-07T19:29:07.916 INFO:teuthology.provision.fog.smithi098:Deploy complete!
2024-03-07T19:29:07.918 DEBUG:teuthology.parallel:result is None
2024-03-07T19:29:53.876 WARNING:teuthology.provision.fog.smithi040:timed out
2024-03-07T19:29:59.878 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:30:07.014 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40
2024-03-07T19:30:13.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:30:13.017 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40
2024-03-07T19:30:19.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:30:19.020 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40
2024-03-07T19:30:25.021 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T19:30:40.579 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-07T19:30:40.579 DEBUG:teuthology.orchestra.run.smithi040:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-07T19:30:41.032 INFO:teuthology.provision.fog.smithi040:Node is ready
2024-03-07T19:30:41.032 DEBUG:teuthology.orchestra.run.smithi040:> hostname
2024-03-07T19:30:41.038 INFO:teuthology.orchestra.run.smithi040.stdout:smithi040.front.sepia.ceph.com
2024-03-07T19:30:41.038 DEBUG:teuthology.orchestra.run.smithi040:> grep smithi040.front.sepia.ceph.com /etc/hosts
2024-03-07T19:30:41.084 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40 smithi040.front.sepia.ceph.com smithi040
2024-03-07T19:30:41.085 DEBUG:teuthology.orchestra.run.smithi040:> sudo hostname smithi040
2024-03-07T19:30:41.213 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i -e 's/smithi040.front.sepia.ceph.com/smithi040/g' /etc/hosts
2024-03-07T19:30:41.228 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i -e 's/172.21.15.40/172.21.15.40/g' /etc/hosts
2024-03-07T19:30:41.285 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i -e 's/smithi040.front.sepia.ceph.com/smithi040/g' /etc/hostname
2024-03-07T19:30:41.340 DEBUG:teuthology.orchestra.run.smithi040:> sudo hostname smithi040
2024-03-07T19:30:41.394 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release
2024-03-07T19:30:41.439 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2024-03-07T19:30:41.440 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="Ubuntu"
2024-03-07T19:30:41.440 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="22.04"
2024-03-07T19:30:41.440 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2024-03-07T19:30:41.440 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_CODENAME=jammy
2024-03-07T19:30:41.440 INFO:teuthology.orchestra.run.smithi040.stdout:ID=ubuntu
2024-03-07T19:30:41.440 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE=debian
2024-03-07T19:30:41.440 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://www.ubuntu.com/"
2024-03-07T19:30:41.440 INFO:teuthology.orchestra.run.smithi040.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-03-07T19:30:41.440 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-03-07T19:30:41.440 INFO:teuthology.orchestra.run.smithi040.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-03-07T19:30:41.440 INFO:teuthology.orchestra.run.smithi040.stdout:UBUNTU_CODENAME=jammy
2024-03-07T19:30:41.442 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/fog.py", line 92, in create
    self._verify_installed_os()
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/fog.py", line 347, in _verify_installed_os
    raise RuntimeError(
RuntimeError: Expected smithi040's OS to be centos 9 but found ubuntu 22.04
2024-03-07T19:30:41.443 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T19:30:41.729 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=4132daa072424ce6a5eebbd06f1c2319
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/fog.py", line 92, in create
    self._verify_installed_os()
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/fog.py", line 347, in _verify_installed_os
    raise RuntimeError(
RuntimeError: Expected smithi040's OS to be centos 9 but found ubuntu 22.04
2024-03-07T19:30:41.731 INFO:teuthology.nuke:Checking targets against current locks
2024-03-07T19:30:41.795 DEBUG:teuthology.nuke:shortname: smithi040
2024-03-07T19:30:41.796 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-03-07T19:30:41.816 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-07_10:41:12-fs-wip-vshankar-testing-20240307.022905-testing-default-smithi/7584524', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 19:24:09.798549', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGhKo59xwNsqe8kZGCulRFlK7h3s7TEQtFaxdhvMa+F4Gyzbl514mwBh2ioaHxh5FTmE1QtZwIG7kR7tiMIXNYA='}
2024-03-07T19:30:41.819 DEBUG:teuthology.nuke:shortname: smithi098
2024-03-07T19:30:41.819 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-03-07T19:30:41.838 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-07_10:41:12-fs-wip-vshankar-testing-20240307.022905-testing-default-smithi/7584523', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 19:24:09.366347', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHTvsyriWxKWgrpFcLvx8jlQl0XQnjHkX9/zurlhLhqslmJARYPvIFi/MSv7KxzOSbPrySBBSrV/Mz6wOl8yqlU='}
2024-03-07T19:30:41.862 INFO:teuthology.orchestra.console.smithi040:Power off
2024-03-07T19:30:41.863 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-07T19:30:41.891 DEBUG:teuthology.orchestra.console.smithi040:power off output: Chassis Power Control: Down/Off
2024-03-07T19:30:41.891 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T19:30:41.907 INFO:teuthology.orchestra.console.smithi098:Power off
2024-03-07T19:30:41.907 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-07T19:30:41.917 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on
2024-03-07T19:30:41.931 DEBUG:teuthology.orchestra.console.smithi098:power off output: Chassis Power Control: Down/Off
2024-03-07T19:30:41.931 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T19:30:41.953 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on
2024-03-07T19:30:45.918 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T19:30:45.954 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T19:30:46.029 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on
2024-03-07T19:30:46.065 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on
2024-03-07T19:30:50.030 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T19:30:50.065 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T19:30:50.142 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on
2024-03-07T19:30:50.175 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off
2024-03-07T19:30:50.276 INFO:teuthology.orchestra.console.smithi098:Power off completed
2024-03-07T19:30:50.444 INFO:teuthology.lock.ops:unlocked: smithi098.front.sepia.ceph.com
2024-03-07T19:30:54.143 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T19:30:54.252 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off
2024-03-07T19:30:54.354 INFO:teuthology.orchestra.console.smithi040:Power off completed
2024-03-07T19:30:54.510 INFO:teuthology.lock.ops:unlocked: smithi040.front.sepia.ceph.com
2024-03-07T19:30:54.757 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/fog.py", line 92, in create
    self._verify_installed_os()
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/provision/fog.py", line 347, in _verify_installed_os
    raise RuntimeError(
RuntimeError: Expected smithi040's OS to be centos 9 but found ubuntu 22.04