2023-11-23T19:54:18.294 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-23T19:54:18.295 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-23T19:54:18.424 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fadaea2ec10>, signals=[15]) 2023-11-23T19:54:18.424 INFO:teuthology.lock.ops:Start node 'smithi132.front.sepia.ceph.com' reimaging 2023-11-23T19:54:18.425 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: reset os type and version on server 2023-11-23T19:54:18.425 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2023-11-23T19:54:18.443 INFO:teuthology.lock.ops:Start node 'smithi150.front.sepia.ceph.com' reimaging 2023-11-23T19:54:18.443 INFO:teuthology.lock.ops:Updating [smithi150.front.sepia.ceph.com]: reset os type and version on server 2023-11-23T19:54:18.443 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2023-11-23T19:54:18.567 DEBUG:teuthology.provision.fog.smithi132:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-11-23T19:54:18.597 DEBUG:teuthology.provision.fog.smithi150:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-11-23T19:54:18.620 INFO:teuthology.provision.fog.smithi132:Scheduling deploy of ubuntu 22.04 2023-11-23T19:54:18.656 INFO:teuthology.provision.fog.smithi150:Scheduling deploy of ubuntu 22.04 2023-11-23T19:54:19.250 INFO:teuthology.orchestra.console.smithi150:Power off 2023-11-23T19:54:19.250 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-23T19:54:19.264 INFO:teuthology.orchestra.console.smithi132:Power off 2023-11-23T19:54:19.264 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-23T19:54:19.270 DEBUG:teuthology.orchestra.console.smithi150:power off output: Chassis Power Control: Down/Off 2023-11-23T19:54:19.271 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-23T19:54:19.284 DEBUG:teuthology.orchestra.console.smithi132:power off output: Chassis Power Control: Down/Off 2023-11-23T19:54:19.284 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-23T19:54:19.291 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2023-11-23T19:54:19.306 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2023-11-23T19:54:23.296 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-23T19:54:23.308 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-23T19:54:23.404 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2023-11-23T19:54:23.416 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2023-11-23T19:54:27.410 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-23T19:54:27.418 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-23T19:54:27.519 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2023-11-23T19:54:27.526 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2023-11-23T19:54:27.621 INFO:teuthology.orchestra.console.smithi150:Power off completed 2023-11-23T19:54:27.627 INFO:teuthology.orchestra.console.smithi132:Power off completed 2023-11-23T19:54:27.721 INFO:teuthology.orchestra.console.smithi150:Power on 2023-11-23T19:54:27.722 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-23T19:54:27.728 INFO:teuthology.orchestra.console.smithi132:Power on 2023-11-23T19:54:27.728 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-23T19:54:27.741 DEBUG:teuthology.orchestra.console.smithi150:power on output: Chassis Power Control: Up/On 2023-11-23T19:54:27.741 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-23T19:54:27.749 DEBUG:teuthology.orchestra.console.smithi132:power on output: Chassis Power Control: Up/On 2023-11-23T19:54:27.750 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-23T19:54:27.759 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2023-11-23T19:54:27.768 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2023-11-23T19:54:31.762 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-23T19:54:31.769 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-23T19:54:31.871 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2023-11-23T19:54:31.876 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2023-11-23T19:54:35.877 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-23T19:54:35.884 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-23T19:54:35.987 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2023-11-23T19:54:35.992 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2023-11-23T19:54:36.088 INFO:teuthology.orchestra.console.smithi150:Power on completed 2023-11-23T19:54:36.093 INFO:teuthology.orchestra.console.smithi132:Power on completed 2023-11-23T19:54:36.189 INFO:teuthology.provision.fog.smithi150:Waiting for deploy to finish 2023-11-23T19:54:36.194 INFO:teuthology.provision.fog.smithi132:Waiting for deploy to finish 2023-11-23T19:54:36.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T19:54:39.525 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-11-23T19:54:45.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T19:54:48.610 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-11-23T19:54:54.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T19:54:57.666 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-11-23T19:55:03.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T19:55:06.754 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-11-23T19:55:12.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T19:55:15.842 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-11-23T19:55:21.849 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T19:55:24.930 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-11-23T19:55:30.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T19:55:34.018 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-11-23T19:55:40.022 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T19:55:58.434 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-11-23T19:56:04.441 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T19:56:07.522 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-11-23T19:56:13.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T19:56:13.657 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T19:56:17.784 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T19:56:24.909 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T19:56:35.032 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T19:56:48.156 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T19:57:04.281 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T19:57:23.406 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T19:57:45.530 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T19:58:10.655 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T19:58:38.778 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T19:58:38.778 WARNING:teuthology.provision.fog.smithi132:'connect to smithi132.front.sepia.ceph.com' reached maximum tries (11) after waiting for 145 seconds 2023-11-23T19:58:44.779 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T19:58:44.901 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T19:58:49.025 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T19:58:56.148 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T19:59:06.274 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T19:59:19.398 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T19:59:35.483 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T19:59:43.837 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T19:59:46.914 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2023-11-23T19:59:52.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T19:59:54.567 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T19:59:55.970 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2023-11-23T20:00:01.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:00:05.026 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2023-11-23T20:00:11.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:00:14.082 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2023-11-23T20:00:16.650 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T20:00:20.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:00:23.138 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2023-11-23T20:00:29.140 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:00:32.194 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2023-11-23T20:00:38.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:00:41.250 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2023-11-23T20:00:41.735 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T20:00:47.252 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:01:09.859 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-23T20:01:09.859 WARNING:teuthology.provision.fog.smithi132:'connect to smithi132.front.sepia.ceph.com' reached maximum tries (11) after waiting for 145 seconds 2023-11-23T20:01:15.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:01:47.255 WARNING:teuthology.provision.fog.smithi150:timed out 2023-11-23T20:01:53.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:01:53.258 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2023-11-23T20:01:59.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:01:59.475 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-23T20:01:59.475 DEBUG:teuthology.orchestra.run.smithi150:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-23T20:01:59.901 INFO:teuthology.provision.fog.smithi150:Node is ready 2023-11-23T20:01:59.901 DEBUG:teuthology.orchestra.run.smithi150:> hostname 2023-11-23T20:01:59.948 INFO:teuthology.orchestra.run.smithi150.stdout:smithi150.front.sepia.ceph.com 2023-11-23T20:01:59.949 DEBUG:teuthology.orchestra.run.smithi150:> grep smithi150.front.sepia.ceph.com /etc/hosts 2023-11-23T20:01:59.996 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150 smithi150.front.sepia.ceph.com smithi150 2023-11-23T20:01:59.997 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname smithi150 2023-11-23T20:02:00.145 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i -e 's/smithi150.front.sepia.ceph.com/smithi150/g' /etc/hosts 2023-11-23T20:02:00.160 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i -e 's/172.21.15.150/172.21.15.150/g' /etc/hosts 2023-11-23T20:02:00.216 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i -e 's/smithi150.front.sepia.ceph.com/smithi150/g' /etc/hostname 2023-11-23T20:02:00.273 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname smithi150 2023-11-23T20:02:00.328 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2023-11-23T20:02:00.376 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-23T20:02:00.377 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="Ubuntu" 2023-11-23T20:02:00.377 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="22.04" 2023-11-23T20:02:00.377 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-23T20:02:00.377 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_CODENAME=jammy 2023-11-23T20:02:00.377 INFO:teuthology.orchestra.run.smithi150.stdout:ID=ubuntu 2023-11-23T20:02:00.378 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE=debian 2023-11-23T20:02:00.378 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-23T20:02:00.378 INFO:teuthology.orchestra.run.smithi150.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-23T20:02:00.378 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-23T20:02:00.378 INFO:teuthology.orchestra.run.smithi150.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-23T20:02:00.378 INFO:teuthology.orchestra.run.smithi150.stdout:UBUNTU_CODENAME=jammy 2023-11-23T20:02:00.379 INFO:teuthology.provision.fog.smithi150:Deploy complete! 2023-11-23T20:02:00.380 DEBUG:teuthology.parallel:result is None 2023-11-23T20:02:15.863 WARNING:teuthology.provision.fog.smithi132:timed out 2023-11-23T20:02:21.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:02:24.934 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-11-23T20:02:30.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:02:33.990 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-11-23T20:02:39.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:02:40.134 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-11-23T20:02:46.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:02:47.138 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-11-23T20:02:53.140 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:02:53.142 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-11-23T20:02:59.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:02:59.145 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-11-23T20:03:05.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:03:05.148 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-11-23T20:03:11.149 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:03:11.151 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-11-23T20:03:17.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:03:17.227 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-23T20:03:17.227 DEBUG:teuthology.orchestra.run.smithi132:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-23T20:03:17.338 INFO:teuthology.provision.fog.smithi132:Node is ready 2023-11-23T20:03:17.338 DEBUG:teuthology.orchestra.run.smithi132:> hostname 2023-11-23T20:03:17.384 INFO:teuthology.orchestra.run.smithi132.stdout:smithi132 2023-11-23T20:03:17.385 DEBUG:teuthology.orchestra.run.smithi132:> grep smithi132 /etc/hosts 2023-11-23T20:03:17.432 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132 smithi132 smithi132 2023-11-23T20:03:17.432 DEBUG:teuthology.orchestra.run.smithi132:> sudo hostname smithi132 2023-11-23T20:03:17.486 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i -e 's/smithi132/smithi132/g' /etc/hosts 2023-11-23T20:03:17.539 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i -e 's/172.21.15.132/172.21.15.132/g' /etc/hosts 2023-11-23T20:03:17.623 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i -e 's/smithi132/smithi132/g' /etc/hostname 2023-11-23T20:03:17.681 DEBUG:teuthology.orchestra.run.smithi132:> sudo hostname smithi132 2023-11-23T20:03:17.734 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2023-11-23T20:03:17.781 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-23T20:03:17.781 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="Ubuntu" 2023-11-23T20:03:17.782 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="22.04" 2023-11-23T20:03:17.782 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-23T20:03:17.782 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_CODENAME=jammy 2023-11-23T20:03:17.782 INFO:teuthology.orchestra.run.smithi132.stdout:ID=ubuntu 2023-11-23T20:03:17.782 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE=debian 2023-11-23T20:03:17.782 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-23T20:03:17.782 INFO:teuthology.orchestra.run.smithi132.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-23T20:03:17.782 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-23T20:03:17.782 INFO:teuthology.orchestra.run.smithi132.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-23T20:03:17.782 INFO:teuthology.orchestra.run.smithi132.stdout:UBUNTU_CODENAME=jammy 2023-11-23T20:03:17.783 INFO:teuthology.provision.fog.smithi132:Deploy complete! 2023-11-23T20:03:17.783 DEBUG:teuthology.parallel:result is None 2023-11-23T20:03:17.970 INFO:teuthology.lock.ops:Checking smithi132.front.sepia.ceph.com 2023-11-23T20:03:17.971 INFO:teuthology.lock.ops:Checking smithi150.front.sepia.ceph.com 2023-11-23T20:03:17.971 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-23T20:03:17.998 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: set os type and version on server 2023-11-23T20:03:17.999 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-11-23T20:03:17.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:03:18.091 DEBUG:teuthology.orchestra.run.smithi132:> true 2023-11-23T20:03:18.189 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-11-23T20:03:18.190 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2023-11-23T20:03:18.235 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2023-11-23T20:03:18.236 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2023-11-23T20:03:18.283 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-23T20:03:18.283 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="Ubuntu" 2023-11-23T20:03:18.284 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="22.04" 2023-11-23T20:03:18.284 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-23T20:03:18.284 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_CODENAME=jammy 2023-11-23T20:03:18.284 INFO:teuthology.orchestra.run.smithi132.stdout:ID=ubuntu 2023-11-23T20:03:18.285 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE=debian 2023-11-23T20:03:18.285 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-23T20:03:18.285 INFO:teuthology.orchestra.run.smithi132.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-23T20:03:18.285 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-23T20:03:18.286 INFO:teuthology.orchestra.run.smithi132.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-23T20:03:18.286 INFO:teuthology.orchestra.run.smithi132.stdout:UBUNTU_CODENAME=jammy 2023-11-23T20:03:18.286 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2023-11-23T20:03:18.305 INFO:teuthology.lock.ops:Updating [smithi150.front.sepia.ceph.com]: set os type and version on server 2023-11-23T20:03:18.306 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-11-23T20:03:18.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-23T20:03:18.387 DEBUG:teuthology.orchestra.run.smithi150:> true 2023-11-23T20:03:18.610 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-11-23T20:03:18.610 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2023-11-23T20:03:18.657 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2023-11-23T20:03:18.657 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2023-11-23T20:03:18.703 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-23T20:03:18.703 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="Ubuntu" 2023-11-23T20:03:18.703 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="22.04" 2023-11-23T20:03:18.704 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-23T20:03:18.704 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_CODENAME=jammy 2023-11-23T20:03:18.704 INFO:teuthology.orchestra.run.smithi150.stdout:ID=ubuntu 2023-11-23T20:03:18.704 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE=debian 2023-11-23T20:03:18.704 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-23T20:03:18.704 INFO:teuthology.orchestra.run.smithi150.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-23T20:03:18.704 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-23T20:03:18.705 INFO:teuthology.orchestra.run.smithi150.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-23T20:03:18.705 INFO:teuthology.orchestra.run.smithi150.stdout:UBUNTU_CODENAME=jammy 2023-11-23T20:03:18.705 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2023-11-23T20:03:18.730 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-23T20:03:18.810 INFO:teuthology.dispatcher.supervisor:Running job 7465260 2023-11-23T20:03:18.810 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/virtualenv/bin/teuthology -v --owner scheduled_cbodley@teuthology --archive /home/teuthworker/archive/cbodley-2023-11-23_19:53:09-rgw-wip-cbodley-testing-distro-default-smithi/7465260 --name cbodley-2023-11-23_19:53:09-rgw-wip-cbodley-testing-distro-default-smithi --description rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} -- /home/teuthworker/archive/cbodley-2023-11-23_19:53:09-rgw-wip-cbodley-testing-distro-default-smithi/7465260/orig.config.yaml 2023-11-23T20:03:18.814 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/cbodley-2023-11-23_19:53:09-rgw-wip-cbodley-testing-distro-default-smithi/7465260 2023-11-23T20:03:18.815 INFO:teuthology.dispatcher.supervisor:Job PID: 2546759 2023-11-23T20:03:18.815 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-23T20:05:18.816 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-23T20:05:18.885 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-11-23T20:05:19.009 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi132 but it was locked by another job: /home/teuthworker/archive/cbodley-2023-11-23_19:53:09-rgw-wip-cbodley-testing-distro-default-smithi/7465261 2023-11-23T20:05:19.009 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-23T20:05:19.009 INFO:teuthology.nuke:Checking targets against current locks 2023-11-23T20:05:19.066 DEBUG:teuthology.nuke:shortname: smithi132 2023-11-23T20:05:19.067 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-23T20:05:19.089 DEBUG:teuthology.nuke:shortname: smithi150 2023-11-23T20:05:19.089 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-23T20:05:19.090 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2023-11-23_19:53:09-rgw-wip-cbodley-testing-distro-default-smithi/7465261', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-23 19:54:17.790954', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPA4uU60expII9s49C+ybZYvB79rcNs57ucI/S2/d0McWKuQdYvi2Pm6yhvnrvVv8cwGOvkmMuPXJGFkJJL2MFY='} 2023-11-23T20:05:19.116 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2023-11-23_19:53:09-rgw-wip-cbodley-testing-distro-default-smithi/7465260', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-23 19:54:17.436969', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyhXbOcY0+8JqHjTqdzqSov1f89CoO8FFL638/QBimmgRqgo1OQvZoCbV+0vaw26QsjyKWhtDGbBX6RdkvsWQ4='} 2023-11-23T20:05:19.143 INFO:teuthology.orchestra.console.smithi132:Power off 2023-11-23T20:05:19.143 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-23T20:05:19.168 DEBUG:teuthology.orchestra.console.smithi132:power off output: Chassis Power Control: Down/Off 2023-11-23T20:05:19.168 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-23T20:05:19.194 INFO:teuthology.orchestra.console.smithi150:Power off 2023-11-23T20:05:19.195 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-23T20:05:19.208 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2023-11-23T20:05:19.227 DEBUG:teuthology.orchestra.console.smithi150:power off output: Chassis Power Control: Down/Off 2023-11-23T20:05:19.227 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-23T20:05:19.266 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2023-11-23T20:05:19.309 INFO:teuthology.orchestra.console.smithi132:Power off completed 2023-11-23T20:05:19.367 INFO:teuthology.orchestra.console.smithi150:Power off completed 2023-11-23T20:05:19.449 ERROR:teuthology.lock.ops:failed to unlock smithi132.front.sepia.ceph.com. reason: Cannot unlock an already-unlocked node 2023-11-23T20:05:19.516 INFO:teuthology.lock.ops:unlocked: smithi150.front.sepia.ceph.com