2023-12-10T07:55:24.286 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-10T07:55:24.287 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T07:55:24.423 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8b03c05550>, signals=[15]) 2023-12-10T07:55:24.424 INFO:teuthology.lock.ops:Start node 'smithi174.front.sepia.ceph.com' reimaging 2023-12-10T07:55:24.424 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: reset os type and version on server 2023-12-10T07:55:24.424 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2023-12-10T07:55:24.446 INFO:teuthology.lock.ops:Start node 'smithi178.front.sepia.ceph.com' reimaging 2023-12-10T07:55:24.446 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: reset os type and version on server 2023-12-10T07:55:24.446 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2023-12-10T07:55:24.543 DEBUG:teuthology.provision.fog.smithi174:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-12-10T07:55:24.571 DEBUG:teuthology.provision.fog.smithi178:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-12-10T07:55:24.583 INFO:teuthology.provision.fog.smithi174:Scheduling deploy of ubuntu 22.04 2023-12-10T07:55:24.615 INFO:teuthology.provision.fog.smithi178:Scheduling deploy of ubuntu 22.04 2023-12-10T07:55:25.070 INFO:teuthology.orchestra.console.smithi174:Power off 2023-12-10T07:55:25.070 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-10T07:55:25.081 INFO:teuthology.orchestra.console.smithi178:Power off 2023-12-10T07:55:25.082 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-10T07:55:25.091 DEBUG:teuthology.orchestra.console.smithi174:power off output: Chassis Power Control: Down/Off 2023-12-10T07:55:25.091 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T07:55:25.102 DEBUG:teuthology.orchestra.console.smithi178:power off output: Chassis Power Control: Down/Off 2023-12-10T07:55:25.102 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T07:55:25.111 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off 2023-12-10T07:55:25.123 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2023-12-10T07:55:25.212 INFO:teuthology.orchestra.console.smithi174:Power off completed 2023-12-10T07:55:25.313 INFO:teuthology.orchestra.console.smithi174:Power on 2023-12-10T07:55:25.313 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-10T07:55:25.337 DEBUG:teuthology.orchestra.console.smithi174:power on output: Chassis Power Control: Up/On 2023-12-10T07:55:25.337 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T07:55:25.358 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off 2023-12-10T07:55:29.126 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T07:55:29.235 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2023-12-10T07:55:29.359 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T07:55:29.468 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2023-12-10T07:55:29.569 INFO:teuthology.orchestra.console.smithi174:Power on completed 2023-12-10T07:55:29.669 INFO:teuthology.provision.fog.smithi174:Waiting for deploy to finish 2023-12-10T07:55:33.238 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T07:55:33.347 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2023-12-10T07:55:33.447 INFO:teuthology.orchestra.console.smithi178:Power off completed 2023-12-10T07:55:33.548 INFO:teuthology.orchestra.console.smithi178:Power on 2023-12-10T07:55:33.548 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-10T07:55:33.567 DEBUG:teuthology.orchestra.console.smithi178:power on output: Chassis Power Control: Up/On 2023-12-10T07:55:33.567 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T07:55:33.587 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2023-12-10T07:55:37.592 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T07:55:37.700 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2023-12-10T07:55:41.705 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T07:55:41.906 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2023-12-10T07:55:42.006 INFO:teuthology.orchestra.console.smithi178:Power on completed 2023-12-10T07:55:42.107 INFO:teuthology.provision.fog.smithi178:Waiting for deploy to finish 2023-12-10T08:03:48.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:03:51.074 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-12-10T08:03:57.081 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:04:00.166 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-12-10T08:04:06.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:04:09.250 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-12-10T08:04:15.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:04:18.306 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-12-10T08:04:24.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:04:27.394 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-12-10T08:04:33.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:04:36.450 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-12-10T08:04:42.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:05:00.898 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-12-10T08:05:06.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:05:09.954 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-12-10T08:05:15.958 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:05:19.010 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-12-10T08:05:25.014 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:05:25.017 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-12-10T08:05:31.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:05:31.022 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-12-10T08:05:37.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:05:37.030 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-12-10T08:05:43.036 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:05:43.038 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-12-10T08:05:43.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:05:49.041 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:05:49.042 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-12-10T08:05:55.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:05:55.361 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-10T08:05:55.362 DEBUG:teuthology.orchestra.run.smithi174:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-10T08:05:55.824 INFO:teuthology.provision.fog.smithi174:Node is ready 2023-12-10T08:05:55.825 DEBUG:teuthology.orchestra.run.smithi174:> hostname 2023-12-10T08:05:55.872 INFO:teuthology.orchestra.run.smithi174.stdout:smithi174.front.sepia.ceph.com 2023-12-10T08:05:55.872 DEBUG:teuthology.orchestra.run.smithi174:> grep smithi174.front.sepia.ceph.com /etc/hosts 2023-12-10T08:05:55.921 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.15.174 smithi174.front.sepia.ceph.com smithi174 2023-12-10T08:05:55.921 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname smithi174 2023-12-10T08:05:56.086 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/smithi174.front.sepia.ceph.com/smithi174/g' /etc/hosts 2023-12-10T08:05:56.099 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/172.21.15.174/172.21.15.174/g' /etc/hosts 2023-12-10T08:05:56.157 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/smithi174.front.sepia.ceph.com/smithi174/g' /etc/hostname 2023-12-10T08:05:56.213 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname smithi174 2023-12-10T08:05:56.272 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2023-12-10T08:05:56.321 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-12-10T08:05:56.321 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="Ubuntu" 2023-12-10T08:05:56.321 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="22.04" 2023-12-10T08:05:56.321 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-12-10T08:05:56.321 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_CODENAME=jammy 2023-12-10T08:05:56.321 INFO:teuthology.orchestra.run.smithi174.stdout:ID=ubuntu 2023-12-10T08:05:56.321 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE=debian 2023-12-10T08:05:56.321 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-10T08:05:56.321 INFO:teuthology.orchestra.run.smithi174.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-10T08:05:56.321 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-10T08:05:56.321 INFO:teuthology.orchestra.run.smithi174.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-10T08:05:56.322 INFO:teuthology.orchestra.run.smithi174.stdout:UBUNTU_CODENAME=jammy 2023-12-10T08:05:56.322 INFO:teuthology.provision.fog.smithi174:Deploy complete! 2023-12-10T08:05:56.323 DEBUG:teuthology.parallel:result is None 2023-12-10T08:07:14.562 ERROR:teuthology.orchestra.connection:Error authenticating with smithi178.front.sepia.ceph.com: AuthenticationException('Authentication timeout.') 2023-12-10T08:07:28.866 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-12-10T08:07:34.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:07:36.898 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-12-10T08:07:42.901 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:07:42.903 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-12-10T08:07:48.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:07:48.906 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-12-10T08:07:54.907 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:07:54.909 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-12-10T08:07:59.330 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-12-10T08:08:00.916 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:08:01.192 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-10T08:08:01.192 DEBUG:teuthology.orchestra.run.smithi178:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-10T08:08:01.630 INFO:teuthology.provision.fog.smithi178:Node is ready 2023-12-10T08:08:01.630 DEBUG:teuthology.orchestra.run.smithi178:> hostname 2023-12-10T08:08:01.678 INFO:teuthology.orchestra.run.smithi178.stdout:smithi178.front.sepia.ceph.com 2023-12-10T08:08:01.679 DEBUG:teuthology.orchestra.run.smithi178:> grep smithi178.front.sepia.ceph.com /etc/hosts 2023-12-10T08:08:01.731 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178 smithi178.front.sepia.ceph.com smithi178 2023-12-10T08:08:01.731 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname smithi178 2023-12-10T08:08:01.880 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i -e 's/smithi178.front.sepia.ceph.com/smithi178/g' /etc/hosts 2023-12-10T08:08:01.895 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i -e 's/172.21.15.178/172.21.15.178/g' /etc/hosts 2023-12-10T08:08:01.950 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i -e 's/smithi178.front.sepia.ceph.com/smithi178/g' /etc/hostname 2023-12-10T08:08:02.006 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname smithi178 2023-12-10T08:08:02.061 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2023-12-10T08:08:02.111 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-12-10T08:08:02.111 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Ubuntu" 2023-12-10T08:08:02.111 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="22.04" 2023-12-10T08:08:02.111 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-12-10T08:08:02.111 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_CODENAME=jammy 2023-12-10T08:08:02.112 INFO:teuthology.orchestra.run.smithi178.stdout:ID=ubuntu 2023-12-10T08:08:02.112 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE=debian 2023-12-10T08:08:02.112 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-10T08:08:02.112 INFO:teuthology.orchestra.run.smithi178.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-10T08:08:02.112 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-10T08:08:02.112 INFO:teuthology.orchestra.run.smithi178.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-10T08:08:02.112 INFO:teuthology.orchestra.run.smithi178.stdout:UBUNTU_CODENAME=jammy 2023-12-10T08:08:02.112 INFO:teuthology.provision.fog.smithi178:Deploy complete! 2023-12-10T08:08:02.113 DEBUG:teuthology.parallel:result is None 2023-12-10T08:08:02.339 INFO:teuthology.lock.ops:Checking smithi174.front.sepia.ceph.com 2023-12-10T08:08:02.340 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-10T08:08:02.363 INFO:teuthology.lock.ops:Checking smithi178.front.sepia.ceph.com 2023-12-10T08:08:02.363 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-10T08:08:02.390 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: set os type and version on server 2023-12-10T08:08:02.390 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2023-12-10T08:08:02.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:08:02.531 DEBUG:teuthology.orchestra.run.smithi174:> true 2023-12-10T08:08:02.746 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2023-12-10T08:08:02.746 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2023-12-10T08:08:02.798 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2023-12-10T08:08:02.798 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2023-12-10T08:08:02.851 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-12-10T08:08:02.851 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="Ubuntu" 2023-12-10T08:08:02.852 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="22.04" 2023-12-10T08:08:02.852 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-12-10T08:08:02.852 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_CODENAME=jammy 2023-12-10T08:08:02.852 INFO:teuthology.orchestra.run.smithi174.stdout:ID=ubuntu 2023-12-10T08:08:02.852 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE=debian 2023-12-10T08:08:02.852 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-10T08:08:02.852 INFO:teuthology.orchestra.run.smithi174.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-10T08:08:02.852 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-10T08:08:02.852 INFO:teuthology.orchestra.run.smithi174.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-10T08:08:02.852 INFO:teuthology.orchestra.run.smithi174.stdout:UBUNTU_CODENAME=jammy 2023-12-10T08:08:02.853 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2023-12-10T08:08:02.885 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: set os type and version on server 2023-12-10T08:08:02.885 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-12-10T08:08:02.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-10T08:08:02.963 DEBUG:teuthology.orchestra.run.smithi178:> true 2023-12-10T08:08:03.063 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-12-10T08:08:03.063 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2023-12-10T08:08:03.112 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2023-12-10T08:08:03.112 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2023-12-10T08:08:03.175 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-12-10T08:08:03.176 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Ubuntu" 2023-12-10T08:08:03.176 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="22.04" 2023-12-10T08:08:03.176 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-12-10T08:08:03.176 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_CODENAME=jammy 2023-12-10T08:08:03.176 INFO:teuthology.orchestra.run.smithi178.stdout:ID=ubuntu 2023-12-10T08:08:03.176 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE=debian 2023-12-10T08:08:03.176 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-10T08:08:03.176 INFO:teuthology.orchestra.run.smithi178.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-10T08:08:03.176 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-10T08:08:03.176 INFO:teuthology.orchestra.run.smithi178.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-10T08:08:03.176 INFO:teuthology.orchestra.run.smithi178.stdout:UBUNTU_CODENAME=jammy 2023-12-10T08:08:03.177 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2023-12-10T08:08:03.392 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:08:03.507 INFO:teuthology.dispatcher.supervisor:Running job 7485829 2023-12-10T08:08:03.507 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/bin/teuthology -v --owner scheduled_alimasa@teuthology --archive /home/teuthworker/archive/alimasa-2023-12-10_07:54:38-rgw-wip-alimasa-multi-pubsub-etag-test-distro-default-smithi/7485829 --name alimasa-2023-12-10_07:54:38-rgw-wip-alimasa-multi-pubsub-etag-test-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{ubuntu_latest} tasks/test_multi} -- /home/teuthworker/archive/alimasa-2023-12-10_07:54:38-rgw-wip-alimasa-multi-pubsub-etag-test-distro-default-smithi/7485829/orig.config.yaml 2023-12-10T08:08:03.518 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/alimasa-2023-12-10_07:54:38-rgw-wip-alimasa-multi-pubsub-etag-test-distro-default-smithi/7485829 2023-12-10T08:08:03.519 INFO:teuthology.dispatcher.supervisor:Job PID: 904291 2023-12-10T08:08:03.519 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-10T08:10:03.619 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:12:03.821 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:14:03.988 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:16:04.162 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:18:04.344 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:20:04.527 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:22:04.703 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:24:04.883 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:26:05.055 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:28:05.210 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:30:05.382 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:32:05.558 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:34:05.670 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:36:05.835 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:38:05.961 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:40:06.117 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:42:06.291 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:44:06.449 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:46:06.611 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:48:06.756 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:50:06.931 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:52:07.144 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:54:07.303 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:56:07.460 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T08:58:07.618 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:00:07.777 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:02:07.933 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:04:08.053 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:06:08.219 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:08:08.401 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:10:08.563 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:12:08.669 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:14:08.838 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:16:08.985 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:18:09.138 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:20:09.289 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:22:09.528 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:24:09.703 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:26:09.875 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:28:10.103 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:30:10.363 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:32:10.616 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:34:10.752 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:36:10.947 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:38:11.114 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:40:11.247 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:42:11.415 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:44:11.578 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:46:11.713 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:48:11.865 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:50:12.049 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:52:12.222 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:54:12.388 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:56:12.535 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T09:58:12.691 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T10:00:12.797 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T10:02:12.995 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T10:04:13.161 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T10:06:13.300 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T10:08:13.473 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T10:10:13.604 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T10:12:13.835 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T10:14:14.012 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T10:16:14.167 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T10:18:14.302 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T10:20:14.436 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T10:22:14.559 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T10:24:14.709 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-10T10:24:14.865 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-10T10:24:14.960 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-10T10:24:14.961 INFO:teuthology.nuke:Checking targets against current locks 2023-12-10T10:24:15.003 DEBUG:teuthology.nuke:shortname: smithi174 2023-12-10T10:24:15.003 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-10T10:24:15.020 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/alimasa-2023-12-10_07:54:38-rgw-wip-alimasa-multi-pubsub-etag-test-distro-default-smithi/7485829', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-10 07:55:23.519893', 'locked_by': 'scheduled_alimasa@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB1NyOJmcf8KAqb2e++09sGHLvPBjtRGN0mU50M61bzINa0gMjF2wShOwZauo1TqREfDcwhF+A+L3hCjxRqKYHM='} 2023-12-10T10:24:15.026 DEBUG:teuthology.nuke:shortname: smithi178 2023-12-10T10:24:15.027 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-10T10:24:15.046 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/alimasa-2023-12-10_07:54:38-rgw-wip-alimasa-multi-pubsub-etag-test-distro-default-smithi/7485829', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-10 07:55:23.521626', 'locked_by': 'scheduled_alimasa@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOpBnoDpEeK0KCtn1xLC8MKPv6TJcs4yA7fZ59FwHx3VngUisuwZUGCQ6tybkplR2IYYWZLXUmA89D9x8yisEnY='} 2023-12-10T10:24:15.064 INFO:teuthology.orchestra.console.smithi174:Power off 2023-12-10T10:24:15.065 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-10T10:24:15.088 DEBUG:teuthology.orchestra.console.smithi174:power off output: Chassis Power Control: Down/Off 2023-12-10T10:24:15.089 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T10:24:15.103 INFO:teuthology.orchestra.console.smithi178:Power off 2023-12-10T10:24:15.104 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-10T10:24:15.111 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off 2023-12-10T10:24:15.125 DEBUG:teuthology.orchestra.console.smithi178:power off output: Chassis Power Control: Down/Off 2023-12-10T10:24:15.126 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-10T10:24:15.146 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2023-12-10T10:24:15.213 INFO:teuthology.orchestra.console.smithi174:Power off completed 2023-12-10T10:24:15.247 INFO:teuthology.orchestra.console.smithi178:Power off completed 2023-12-10T10:24:15.369 INFO:teuthology.lock.ops:unlocked: smithi174.front.sepia.ceph.com 2023-12-10T10:24:15.394 INFO:teuthology.lock.ops:unlocked: smithi178.front.sepia.ceph.com