2023-11-13T15:21:58.863 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2
2023-11-13T15:21:58.864 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-13T15:21:58.992 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7ff28e096ac0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7ff28de3fb80>, signals=[15])
2023-11-13T15:21:58.993 INFO:teuthology.lock.ops:Start node 'smithi081.front.sepia.ceph.com' reimaging
2023-11-13T15:21:58.994 INFO:teuthology.lock.ops:Updating [smithi081.front.sepia.ceph.com]: reset os type and version on server
2023-11-13T15:21:58.994 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server
2023-11-13T15:21:59.017 INFO:teuthology.lock.ops:Start node 'smithi164.front.sepia.ceph.com' reimaging
2023-11-13T15:21:59.017 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: reset os type and version on server
2023-11-13T15:21:59.018 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server
2023-11-13T15:21:59.019 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /host {"name": "smithi081"}
2023-11-13T15:21:59.042 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /host {"name": "smithi164"}
2023-11-13T15:21:59.079 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /image {"name": "smithi_centos_9.stream"}
2023-11-13T15:21:59.107 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /image {"name": "smithi_centos_9.stream"}
2023-11-13T15:21:59.121 DEBUG:teuthology.provision.fog.smithi081:Requesting image smithi_centos_9.stream (ID 10)
2023-11-13T15:21:59.121 DEBUG:teuthology.provision.fog.smithi081:FOG request: PUT /host/186 {"imageID": 10}
2023-11-13T15:21:59.146 DEBUG:teuthology.provision.fog.smithi164:Requesting image smithi_centos_9.stream (ID 10)
2023-11-13T15:21:59.146 DEBUG:teuthology.provision.fog.smithi164:FOG request: PUT /host/269 {"imageID": 10}
2023-11-13T15:21:59.163 INFO:teuthology.provision.fog.smithi081:Scheduling deploy of centos 9.stream
2023-11-13T15:21:59.163 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-11-13T15:21:59.189 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of centos 9.stream
2023-11-13T15:21:59.189 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-11-13T15:21:59.256 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /tasktype {"name": "deploy"}
2023-11-13T15:21:59.293 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /tasktype {"name": "deploy"}
2023-11-13T15:21:59.302 DEBUG:teuthology.provision.fog.smithi081:FOG request: POST /host/186/task {"taskTypeID": 1}
2023-11-13T15:21:59.333 DEBUG:teuthology.provision.fog.smithi164:FOG request: POST /host/269/task {"taskTypeID": 1}
2023-11-13T15:21:59.351 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-11-13T15:21:59.387 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-11-13T15:21:59.512 INFO:teuthology.orchestra.console.smithi081:Power off
2023-11-13T15:21:59.513 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-13T15:21:59.534 DEBUG:teuthology.orchestra.console.smithi081:power off output: Chassis Power Control: Down/Off
2023-11-13T15:21:59.534 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:21:59.552 INFO:teuthology.orchestra.console.smithi164:Power off
2023-11-13T15:21:59.553 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-13T15:21:59.560 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on
2023-11-13T15:21:59.574 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off
2023-11-13T15:21:59.574 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:21:59.594 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2023-11-13T15:22:03.566 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:22:03.595 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:22:03.674 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on
2023-11-13T15:22:03.703 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2023-11-13T15:22:07.679 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:22:07.704 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:22:07.790 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on
2023-11-13T15:22:07.814 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2023-11-13T15:22:07.914 INFO:teuthology.orchestra.console.smithi164:Power off completed
2023-11-13T15:22:08.015 INFO:teuthology.orchestra.console.smithi164:Power on
2023-11-13T15:22:08.016 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-13T15:22:08.040 DEBUG:teuthology.orchestra.console.smithi164:power on output: Chassis Power Control: Up/On
2023-11-13T15:22:08.041 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:22:08.063 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2023-11-13T15:22:11.793 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:22:11.905 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is off
2023-11-13T15:22:12.006 INFO:teuthology.orchestra.console.smithi081:Power off completed
2023-11-13T15:22:12.064 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:22:12.106 INFO:teuthology.orchestra.console.smithi081:Power on
2023-11-13T15:22:12.107 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-13T15:22:12.126 DEBUG:teuthology.orchestra.console.smithi081:power on output: Chassis Power Control: Up/On
2023-11-13T15:22:12.126 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:22:12.145 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is off
2023-11-13T15:22:12.171 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2023-11-13T15:22:16.151 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:22:16.172 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:22:16.259 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is off
2023-11-13T15:22:16.327 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2023-11-13T15:22:16.427 INFO:teuthology.orchestra.console.smithi164:Power on completed
2023-11-13T15:22:16.528 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish
2023-11-13T15:22:16.528 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-11-13T15:22:20.264 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:22:20.373 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on
2023-11-13T15:22:20.473 INFO:teuthology.orchestra.console.smithi081:Power on completed
2023-11-13T15:22:20.574 INFO:teuthology.provision.fog.smithi081:Waiting for deploy to finish
2023-11-13T15:22:20.575 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None
2023-11-13T15:22:20.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:22:31.642 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-11-13T15:22:39.099 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-11-13T15:22:45.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:22:46.738 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-11-13T15:22:48.186 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-11-13T15:22:54.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:22:57.274 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-11-13T15:23:01.845 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-11-13T15:23:03.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:23:06.330 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-11-13T15:23:12.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:23:15.418 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-11-13T15:23:16.945 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-11-13T15:23:21.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:23:24.507 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-11-13T15:23:30.518 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:23:32.043 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-11-13T15:23:47.159 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-11-13T15:23:48.987 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-11-13T15:23:54.990 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:23:58.043 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-11-13T15:24:02.305 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-11-13T15:24:04.047 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:24:04.174 ERROR:teuthology.orchestra.connection:Error authenticating with smithi081.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-13T15:24:08.301 ERROR:teuthology.orchestra.connection:Error authenticating with smithi081.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-13T15:24:15.424 ERROR:teuthology.orchestra.connection:Error authenticating with smithi081.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-13T15:24:17.438 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-11-13T15:24:25.548 ERROR:teuthology.orchestra.connection:Error authenticating with smithi081.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-13T15:24:32.586 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-11-13T15:24:38.674 ERROR:teuthology.orchestra.connection:Error authenticating with smithi081.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-13T15:24:47.710 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-11-13T15:24:54.801 ERROR:teuthology.orchestra.connection:Error authenticating with smithi081.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-13T15:25:02.830 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-11-13T15:25:13.927 ERROR:teuthology.orchestra.connection:Error authenticating with smithi081.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-13T15:25:17.961 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-11-13T15:25:18.053 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:25:36.011 ERROR:teuthology.orchestra.connection:Error authenticating with smithi081.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-13T15:25:36.507 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-11-13T15:25:42.509 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:25:45.562 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-11-13T15:25:51.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:25:54.618 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-11-13T15:26:00.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:26:01.137 ERROR:teuthology.orchestra.connection:Error authenticating with smithi081.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-13T15:26:03.674 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-11-13T15:26:09.676 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:26:12.734 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-11-13T15:26:18.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:26:29.262 ERROR:teuthology.orchestra.connection:Error authenticating with smithi081.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-13T15:26:29.262 WARNING:teuthology.provision.fog.smithi081:'connect to smithi081.front.sepia.ceph.com' reached maximum tries (11) after waiting for 145 seconds
2023-11-13T15:26:35.263 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:26:35.388 ERROR:teuthology.orchestra.connection:Error authenticating with smithi081.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-13T15:27:18.740 WARNING:teuthology.provision.fog.smithi164:timed out
2023-11-13T15:27:24.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:27:24.982 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-13T15:27:24.983 DEBUG:teuthology.orchestra.run.smithi164:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-13T15:27:25.317 INFO:teuthology.provision.fog.smithi164:Node is ready
2023-11-13T15:27:25.318 DEBUG:teuthology.orchestra.run.smithi164:> hostname
2023-11-13T15:27:25.332 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164.front.sepia.ceph.com
2023-11-13T15:27:25.332 DEBUG:teuthology.orchestra.run.smithi164:> grep smithi164.front.sepia.ceph.com /etc/hosts
2023-11-13T15:27:25.387 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164 smithi164.front.sepia.ceph.com smithi164
2023-11-13T15:27:25.388 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164
2023-11-13T15:27:25.492 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164.front.sepia.ceph.com/smithi164/g' /etc/hosts
2023-11-13T15:27:25.519 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/172.21.15.164/172.21.15.164/g' /etc/hosts
2023-11-13T15:27:25.586 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164.front.sepia.ceph.com/smithi164/g' /etc/hostname
2023-11-13T15:27:25.653 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164
2023-11-13T15:27:25.720 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release
2023-11-13T15:27:25.775 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream"
2023-11-13T15:27:25.775 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9"
2023-11-13T15:27:25.775 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos"
2023-11-13T15:27:25.775 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora"
2023-11-13T15:27:25.776 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9"
2023-11-13T15:27:25.776 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9"
2023-11-13T15:27:25.776 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9"
2023-11-13T15:27:25.776 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31"
2023-11-13T15:27:25.776 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon"
2023-11-13T15:27:25.777 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-11-13T15:27:25.777 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/"
2023-11-13T15:27:25.777 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-13T15:27:25.777 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-11-13T15:27:25.777 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-11-13T15:27:25.778 INFO:teuthology.provision.fog.smithi164:Deploy complete!
2023-11-13T15:27:25.780 DEBUG:teuthology.parallel:result is None
2023-11-13T15:27:39.391 WARNING:teuthology.provision.fog.smithi081:timed out
2023-11-13T15:27:45.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:28:01.178 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-11-13T15:28:07.180 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:28:10.234 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-11-13T15:28:16.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:28:16.238 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-11-13T15:28:22.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:28:22.242 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-11-13T15:28:28.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:28:28.245 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-11-13T15:28:34.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:28:34.249 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-11-13T15:28:40.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:28:40.252 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2023-11-13T15:28:46.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-13T15:28:46.333 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-13T15:28:46.333 DEBUG:teuthology.orchestra.run.smithi081:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-13T15:28:46.430 INFO:teuthology.provision.fog.smithi081:Node is ready
2023-11-13T15:28:46.431 DEBUG:teuthology.orchestra.run.smithi081:> hostname
2023-11-13T15:28:46.480 INFO:teuthology.orchestra.run.smithi081.stdout:smithi081
2023-11-13T15:28:46.481 DEBUG:teuthology.orchestra.run.smithi081:> grep smithi081 /etc/hosts
2023-11-13T15:28:46.529 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81 smithi081 smithi081
2023-11-13T15:28:46.529 DEBUG:teuthology.orchestra.run.smithi081:> sudo hostname smithi081
2023-11-13T15:28:46.584 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i -e 's/smithi081/smithi081/g' /etc/hosts
2023-11-13T15:28:46.640 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i -e 's/172.21.15.81/172.21.15.81/g' /etc/hosts
2023-11-13T15:28:46.695 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i -e 's/smithi081/smithi081/g' /etc/hostname
2023-11-13T15:28:46.751 DEBUG:teuthology.orchestra.run.smithi081:> sudo hostname smithi081
2023-11-13T15:28:46.807 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release
2023-11-13T15:28:46.852 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-11-13T15:28:46.853 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="Ubuntu"
2023-11-13T15:28:46.853 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="22.04"
2023-11-13T15:28:46.853 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-11-13T15:28:46.853 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_CODENAME=jammy
2023-11-13T15:28:46.854 INFO:teuthology.orchestra.run.smithi081.stdout:ID=ubuntu
2023-11-13T15:28:46.854 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE=debian
2023-11-13T15:28:46.854 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://www.ubuntu.com/"
2023-11-13T15:28:46.854 INFO:teuthology.orchestra.run.smithi081.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-11-13T15:28:46.855 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-11-13T15:28:46.855 INFO:teuthology.orchestra.run.smithi081.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-11-13T15:28:46.855 INFO:teuthology.orchestra.run.smithi081.stdout:UBUNTU_CODENAME=jammy
2023-11-13T15:28:46.857 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 92, in create
    self._verify_installed_os()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 348, in _verify_installed_os
    raise RuntimeError(
RuntimeError: Expected smithi081's OS to be centos 9 but found ubuntu 22.04
2023-11-13T15:28:46.858 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-13T15:28:47.000 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=8cbf8dda420a45b38f1885ef3eb719c3
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 92, in create
    self._verify_installed_os()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 348, in _verify_installed_os
    raise RuntimeError(
RuntimeError: Expected smithi081's OS to be centos 9 but found ubuntu 22.04
2023-11-13T15:28:47.003 INFO:teuthology.nuke:Checking targets against current locks
2023-11-13T15:28:47.072 DEBUG:teuthology.nuke:shortname: smithi081
2023-11-13T15:28:47.072 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-13T15:28:47.093 DEBUG:teuthology.nuke:shortname: smithi164
2023-11-13T15:28:47.093 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-13T15:28:47.097 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rfriedma-2023-11-13_15:17:17-rados:thrash-wip-rf-repl-hp-distro-default-smithi/7456432', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-13 15:21:58.450404', 'locked_by': 'scheduled_rfriedma@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBqlS6TLyPcKKgCNFhgwkFc0IxFjkCxKIXg2RXQvTPye8W9px7A0Pzu6apYfZWPZZsnPwpUMC4zshQhc6GIa/0U='}
2023-11-13T15:28:47.114 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rfriedma-2023-11-13_15:17:17-rados:thrash-wip-rf-repl-hp-distro-default-smithi/7456431', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-13 15:21:58.098464', 'locked_by': 'scheduled_rfriedma@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJBEmhKUMmQJSNg3QdsSU5rBr4ET0VGmHc0WZstM8QVp33/DioCia2V+F6VLESEtV4rn7AIw2AyFlERzwYxw4G4='}
2023-11-13T15:28:47.143 INFO:teuthology.orchestra.console.smithi081:Power off
2023-11-13T15:28:47.144 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-13T15:28:47.159 INFO:teuthology.orchestra.console.smithi164:Power off
2023-11-13T15:28:47.160 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-13T15:28:47.167 DEBUG:teuthology.orchestra.console.smithi081:power off output: Chassis Power Control: Down/Off
2023-11-13T15:28:47.168 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:28:47.183 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off
2023-11-13T15:28:47.183 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:28:47.192 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on
2023-11-13T15:28:47.206 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2023-11-13T15:28:51.194 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:28:51.207 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:28:51.304 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on
2023-11-13T15:28:51.316 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2023-11-13T15:28:55.306 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:28:55.317 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:28:55.416 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on
2023-11-13T15:28:55.427 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2023-11-13T15:28:59.418 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:28:59.428 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-13T15:28:59.528 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is off
2023-11-13T15:28:59.538 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2023-11-13T15:28:59.629 INFO:teuthology.orchestra.console.smithi081:Power off completed
2023-11-13T15:28:59.639 INFO:teuthology.orchestra.console.smithi164:Power off completed
2023-11-13T15:28:59.791 INFO:teuthology.lock.ops:unlocked: smithi081.front.sepia.ceph.com
2023-11-13T15:28:59.819 INFO:teuthology.lock.ops:unlocked: smithi164.front.sepia.ceph.com
2023-11-13T15:29:00.048 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 92, in create
    self._verify_installed_os()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 348, in _verify_installed_os
    raise RuntimeError(
RuntimeError: Expected smithi081's OS to be centos 9 but found ubuntu 22.04