2023-12-02T23:15:49.146 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-02T23:15:49.147 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-02T23:15:49.329 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9f502ad550>, signals=[15]) 2023-12-02T23:15:49.330 INFO:teuthology.lock.ops:Start node 'smithi072.front.sepia.ceph.com' reimaging 2023-12-02T23:15:49.330 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: reset os type and version on server 2023-12-02T23:15:49.330 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2023-12-02T23:15:49.355 INFO:teuthology.lock.ops:Start node 'smithi113.front.sepia.ceph.com' reimaging 2023-12-02T23:15:49.355 INFO:teuthology.lock.ops:Updating [smithi113.front.sepia.ceph.com]: reset os type and version on server 2023-12-02T23:15:49.355 INFO:teuthology.lock.ops:Updating smithi113.front.sepia.ceph.com on lock server 2023-12-02T23:15:49.380 INFO:teuthology.lock.ops:Start node 'smithi156.front.sepia.ceph.com' reimaging 2023-12-02T23:15:49.380 INFO:teuthology.lock.ops:Updating [smithi156.front.sepia.ceph.com]: reset os type and version on server 2023-12-02T23:15:49.380 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2023-12-02T23:15:49.472 DEBUG:teuthology.provision.fog.smithi072:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-12-02T23:15:49.501 DEBUG:teuthology.provision.fog.smithi113:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-12-02T23:15:49.520 DEBUG:teuthology.provision.fog.smithi156:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-12-02T23:15:49.523 INFO:teuthology.provision.fog.smithi072:Scheduling deploy of ubuntu 22.04 2023-12-02T23:15:49.545 INFO:teuthology.provision.fog.smithi113:Scheduling deploy of ubuntu 22.04 2023-12-02T23:15:49.566 INFO:teuthology.provision.fog.smithi156:Scheduling deploy of ubuntu 22.04 2023-12-02T23:15:49.786 INFO:teuthology.orchestra.console.smithi072:Power off 2023-12-02T23:15:49.786 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-02T23:15:49.800 INFO:teuthology.orchestra.console.smithi156:Power off 2023-12-02T23:15:49.800 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-02T23:15:49.811 INFO:teuthology.orchestra.console.smithi113:Power off 2023-12-02T23:15:49.812 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-02T23:15:49.821 DEBUG:teuthology.orchestra.console.smithi072:power off output: Chassis Power Control: Down/Off 2023-12-02T23:15:49.821 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T23:15:49.831 DEBUG:teuthology.orchestra.console.smithi156:power off output: Chassis Power Control: Down/Off 2023-12-02T23:15:49.831 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T23:15:49.840 DEBUG:teuthology.orchestra.console.smithi113:power off output: Chassis Power Control: Down/Off 2023-12-02T23:15:49.840 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T23:15:49.850 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off 2023-12-02T23:15:49.851 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is off 2023-12-02T23:15:49.862 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off 2023-12-02T23:15:49.950 INFO:teuthology.orchestra.console.smithi072:Power off completed 2023-12-02T23:15:49.952 INFO:teuthology.orchestra.console.smithi156:Power off completed 2023-12-02T23:15:49.963 INFO:teuthology.orchestra.console.smithi113:Power off completed 2023-12-02T23:15:50.052 INFO:teuthology.orchestra.console.smithi072:Power on 2023-12-02T23:15:50.052 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-02T23:15:50.061 INFO:teuthology.orchestra.console.smithi156:Power on 2023-12-02T23:15:50.061 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-02T23:15:50.073 INFO:teuthology.orchestra.console.smithi113:Power on 2023-12-02T23:15:50.074 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-02T23:15:50.085 DEBUG:teuthology.orchestra.console.smithi072:power on output: Chassis Power Control: Up/On 2023-12-02T23:15:50.086 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T23:15:50.094 DEBUG:teuthology.orchestra.console.smithi156:power on output: Chassis Power Control: Up/On 2023-12-02T23:15:50.095 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T23:15:50.103 DEBUG:teuthology.orchestra.console.smithi113:power on output: Chassis Power Control: Up/On 2023-12-02T23:15:50.103 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T23:15:50.111 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off 2023-12-02T23:15:50.115 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is off 2023-12-02T23:15:50.123 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off 2023-12-02T23:15:54.115 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T23:15:54.122 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T23:15:54.129 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T23:15:54.224 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2023-12-02T23:15:54.231 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is off 2023-12-02T23:15:54.237 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off 2023-12-02T23:15:54.325 INFO:teuthology.orchestra.console.smithi072:Power on completed 2023-12-02T23:15:54.425 INFO:teuthology.provision.fog.smithi072:Waiting for deploy to finish 2023-12-02T23:15:58.234 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T23:15:58.243 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T23:15:58.353 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2023-12-02T23:15:58.366 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2023-12-02T23:15:58.453 INFO:teuthology.orchestra.console.smithi113:Power on completed 2023-12-02T23:15:58.467 INFO:teuthology.orchestra.console.smithi156:Power on completed 2023-12-02T23:15:58.555 INFO:teuthology.provision.fog.smithi113:Waiting for deploy to finish 2023-12-02T23:15:58.568 INFO:teuthology.provision.fog.smithi156:Waiting for deploy to finish 2023-12-02T23:19:29.662 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:19:29.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:19:32.738 WARNING:teuthology.provision.fog.smithi156:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-12-02T23:19:32.739 WARNING:teuthology.provision.fog.smithi113:[Errno None] Unable to connect to port 22 on 172.21.15.113 2023-12-02T23:19:38.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:19:38.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:19:41.794 WARNING:teuthology.provision.fog.smithi156:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-12-02T23:19:41.830 WARNING:teuthology.provision.fog.smithi113:[Errno None] Unable to connect to port 22 on 172.21.15.113 2023-12-02T23:19:47.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:19:47.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:19:50.850 WARNING:teuthology.provision.fog.smithi156:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-12-02T23:19:50.882 WARNING:teuthology.provision.fog.smithi113:[Errno None] Unable to connect to port 22 on 172.21.15.113 2023-12-02T23:19:56.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:19:56.885 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:19:59.906 WARNING:teuthology.provision.fog.smithi156:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-12-02T23:19:59.938 WARNING:teuthology.provision.fog.smithi113:[Errno None] Unable to connect to port 22 on 172.21.15.113 2023-12-02T23:20:05.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:20:05.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:20:08.962 WARNING:teuthology.provision.fog.smithi156:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-12-02T23:20:08.994 WARNING:teuthology.provision.fog.smithi113:[Errno None] Unable to connect to port 22 on 172.21.15.113 2023-12-02T23:20:10.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:20:14.964 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:20:14.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:20:18.018 WARNING:teuthology.provision.fog.smithi156:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-12-02T23:20:18.050 WARNING:teuthology.provision.fog.smithi113:[Errno None] Unable to connect to port 22 on 172.21.15.113 2023-12-02T23:20:24.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:20:24.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:20:27.106 WARNING:teuthology.provision.fog.smithi113:[Errno None] Unable to connect to port 22 on 172.21.15.113 2023-12-02T23:20:33.108 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:20:36.162 WARNING:teuthology.provision.fog.smithi113:[Errno None] Unable to connect to port 22 on 172.21.15.113 2023-12-02T23:20:42.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:21:10.956 ERROR:teuthology.orchestra.connection:Error authenticating with smithi072.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 427, in connect server_key = t.get_remote_server_key() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 879, in get_remote_server_key raise SSHException("No existing session") paramiko.ssh_exception.SSHException: No existing session 2023-12-02T23:21:24.023 WARNING:teuthology.provision.fog.smithi156:timed out 2023-12-02T23:21:30.024 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:21:30.257 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-02T23:21:30.257 DEBUG:teuthology.orchestra.run.smithi156:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-02T23:21:30.731 INFO:teuthology.provision.fog.smithi156:Node is ready 2023-12-02T23:21:30.731 DEBUG:teuthology.orchestra.run.smithi156:> hostname 2023-12-02T23:21:30.778 INFO:teuthology.orchestra.run.smithi156.stdout:smithi156.front.sepia.ceph.com 2023-12-02T23:21:30.778 DEBUG:teuthology.orchestra.run.smithi156:> grep smithi156.front.sepia.ceph.com /etc/hosts 2023-12-02T23:21:30.827 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.15.156 smithi156.front.sepia.ceph.com smithi156 2023-12-02T23:21:30.827 DEBUG:teuthology.orchestra.run.smithi156:> sudo hostname smithi156 2023-12-02T23:21:30.988 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i -e 's/smithi156.front.sepia.ceph.com/smithi156/g' /etc/hosts 2023-12-02T23:21:31.002 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i -e 's/172.21.15.156/172.21.15.156/g' /etc/hosts 2023-12-02T23:21:31.063 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i -e 's/smithi156.front.sepia.ceph.com/smithi156/g' /etc/hostname 2023-12-02T23:21:31.119 DEBUG:teuthology.orchestra.run.smithi156:> sudo hostname smithi156 2023-12-02T23:21:31.178 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2023-12-02T23:21:31.227 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-12-02T23:21:31.227 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="Ubuntu" 2023-12-02T23:21:31.227 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="22.04" 2023-12-02T23:21:31.227 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-12-02T23:21:31.227 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_CODENAME=jammy 2023-12-02T23:21:31.227 INFO:teuthology.orchestra.run.smithi156.stdout:ID=ubuntu 2023-12-02T23:21:31.227 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE=debian 2023-12-02T23:21:31.227 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-02T23:21:31.227 INFO:teuthology.orchestra.run.smithi156.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-02T23:21:31.228 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-02T23:21:31.228 INFO:teuthology.orchestra.run.smithi156.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-02T23:21:31.228 INFO:teuthology.orchestra.run.smithi156.stdout:UBUNTU_CODENAME=jammy 2023-12-02T23:21:31.228 INFO:teuthology.provision.fog.smithi156:Deploy complete! 2023-12-02T23:21:31.229 DEBUG:teuthology.parallel:result is None 2023-12-02T23:21:42.165 WARNING:teuthology.provision.fog.smithi113:timed out 2023-12-02T23:21:48.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:21:48.384 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-02T23:21:48.384 DEBUG:teuthology.orchestra.run.smithi113:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-02T23:21:48.799 INFO:teuthology.provision.fog.smithi113:Node is ready 2023-12-02T23:21:48.799 DEBUG:teuthology.orchestra.run.smithi113:> hostname 2023-12-02T23:21:48.847 INFO:teuthology.orchestra.run.smithi113.stdout:smithi113.front.sepia.ceph.com 2023-12-02T23:21:48.847 DEBUG:teuthology.orchestra.run.smithi113:> grep smithi113.front.sepia.ceph.com /etc/hosts 2023-12-02T23:21:48.899 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113 smithi113.front.sepia.ceph.com smithi113 2023-12-02T23:21:48.900 DEBUG:teuthology.orchestra.run.smithi113:> sudo hostname smithi113 2023-12-02T23:21:49.039 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i -e 's/smithi113.front.sepia.ceph.com/smithi113/g' /etc/hosts 2023-12-02T23:21:49.052 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i -e 's/172.21.15.113/172.21.15.113/g' /etc/hosts 2023-12-02T23:21:49.110 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i -e 's/smithi113.front.sepia.ceph.com/smithi113/g' /etc/hostname 2023-12-02T23:21:49.170 DEBUG:teuthology.orchestra.run.smithi113:> sudo hostname smithi113 2023-12-02T23:21:49.226 DEBUG:teuthology.orchestra.run.smithi113:> cat /etc/os-release 2023-12-02T23:21:49.275 INFO:teuthology.orchestra.run.smithi113.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-12-02T23:21:49.276 INFO:teuthology.orchestra.run.smithi113.stdout:NAME="Ubuntu" 2023-12-02T23:21:49.276 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_ID="22.04" 2023-12-02T23:21:49.276 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-12-02T23:21:49.276 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_CODENAME=jammy 2023-12-02T23:21:49.276 INFO:teuthology.orchestra.run.smithi113.stdout:ID=ubuntu 2023-12-02T23:21:49.276 INFO:teuthology.orchestra.run.smithi113.stdout:ID_LIKE=debian 2023-12-02T23:21:49.277 INFO:teuthology.orchestra.run.smithi113.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-02T23:21:49.277 INFO:teuthology.orchestra.run.smithi113.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-02T23:21:49.277 INFO:teuthology.orchestra.run.smithi113.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-02T23:21:49.277 INFO:teuthology.orchestra.run.smithi113.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-02T23:21:49.277 INFO:teuthology.orchestra.run.smithi113.stdout:UBUNTU_CODENAME=jammy 2023-12-02T23:21:49.278 INFO:teuthology.provision.fog.smithi113:Deploy complete! 2023-12-02T23:21:49.278 DEBUG:teuthology.parallel:result is None 2023-12-02T23:22:14.961 WARNING:teuthology.provision.fog.smithi072:timed out 2023-12-02T23:22:19.106 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-12-02T23:22:20.964 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:22:20.966 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2023-12-02T23:22:26.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:22:27.221 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-02T23:22:27.222 DEBUG:teuthology.orchestra.run.smithi072:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-02T23:22:27.981 INFO:teuthology.provision.fog.smithi072:Node is ready 2023-12-02T23:22:27.983 DEBUG:teuthology.orchestra.run.smithi072:> hostname 2023-12-02T23:22:28.031 INFO:teuthology.orchestra.run.smithi072.stdout:smithi072.front.sepia.ceph.com 2023-12-02T23:22:28.031 DEBUG:teuthology.orchestra.run.smithi072:> grep smithi072.front.sepia.ceph.com /etc/hosts 2023-12-02T23:22:28.083 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72 smithi072.front.sepia.ceph.com smithi072 2023-12-02T23:22:28.083 DEBUG:teuthology.orchestra.run.smithi072:> sudo hostname smithi072 2023-12-02T23:22:28.316 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i -e 's/smithi072.front.sepia.ceph.com/smithi072/g' /etc/hosts 2023-12-02T23:22:28.331 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i -e 's/172.21.15.72/172.21.15.72/g' /etc/hosts 2023-12-02T23:22:28.386 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i -e 's/smithi072.front.sepia.ceph.com/smithi072/g' /etc/hostname 2023-12-02T23:22:28.442 DEBUG:teuthology.orchestra.run.smithi072:> sudo hostname smithi072 2023-12-02T23:22:28.498 DEBUG:teuthology.orchestra.run.smithi072:> cat /etc/os-release 2023-12-02T23:22:28.546 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-12-02T23:22:28.547 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="Ubuntu" 2023-12-02T23:22:28.547 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="22.04" 2023-12-02T23:22:28.547 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-12-02T23:22:28.547 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_CODENAME=jammy 2023-12-02T23:22:28.547 INFO:teuthology.orchestra.run.smithi072.stdout:ID=ubuntu 2023-12-02T23:22:28.547 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE=debian 2023-12-02T23:22:28.547 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-02T23:22:28.547 INFO:teuthology.orchestra.run.smithi072.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-02T23:22:28.547 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-02T23:22:28.547 INFO:teuthology.orchestra.run.smithi072.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-02T23:22:28.547 INFO:teuthology.orchestra.run.smithi072.stdout:UBUNTU_CODENAME=jammy 2023-12-02T23:22:28.548 INFO:teuthology.provision.fog.smithi072:Deploy complete! 2023-12-02T23:22:28.548 DEBUG:teuthology.parallel:result is None 2023-12-02T23:22:28.776 INFO:teuthology.lock.ops:Checking smithi072.front.sepia.ceph.com 2023-12-02T23:22:28.776 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-02T23:22:28.796 INFO:teuthology.lock.ops:Checking smithi113.front.sepia.ceph.com 2023-12-02T23:22:28.796 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-02T23:22:28.821 INFO:teuthology.lock.ops:Checking smithi156.front.sepia.ceph.com 2023-12-02T23:22:28.821 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-02T23:22:28.855 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: set os type and version on server 2023-12-02T23:22:28.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-12-02T23:22:28.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:22:28.930 DEBUG:teuthology.orchestra.run.smithi072:> true 2023-12-02T23:22:29.038 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-12-02T23:22:29.038 DEBUG:teuthology.orchestra.run.smithi072:> uname -m 2023-12-02T23:22:29.087 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2023-12-02T23:22:29.087 DEBUG:teuthology.orchestra.run.smithi072:> cat /etc/os-release 2023-12-02T23:22:29.135 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-12-02T23:22:29.135 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="Ubuntu" 2023-12-02T23:22:29.135 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="22.04" 2023-12-02T23:22:29.135 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-12-02T23:22:29.135 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_CODENAME=jammy 2023-12-02T23:22:29.135 INFO:teuthology.orchestra.run.smithi072.stdout:ID=ubuntu 2023-12-02T23:22:29.136 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE=debian 2023-12-02T23:22:29.136 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-02T23:22:29.136 INFO:teuthology.orchestra.run.smithi072.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-02T23:22:29.136 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-02T23:22:29.136 INFO:teuthology.orchestra.run.smithi072.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-02T23:22:29.136 INFO:teuthology.orchestra.run.smithi072.stdout:UBUNTU_CODENAME=jammy 2023-12-02T23:22:29.136 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2023-12-02T23:22:29.163 INFO:teuthology.lock.ops:Updating [smithi113.front.sepia.ceph.com]: set os type and version on server 2023-12-02T23:22:29.163 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2023-12-02T23:22:29.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:22:29.237 DEBUG:teuthology.orchestra.run.smithi113:> true 2023-12-02T23:22:29.466 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2023-12-02T23:22:29.466 DEBUG:teuthology.orchestra.run.smithi113:> uname -m 2023-12-02T23:22:29.513 INFO:teuthology.orchestra.run.smithi113.stdout:x86_64 2023-12-02T23:22:29.514 DEBUG:teuthology.orchestra.run.smithi113:> cat /etc/os-release 2023-12-02T23:22:29.562 INFO:teuthology.orchestra.run.smithi113.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-12-02T23:22:29.562 INFO:teuthology.orchestra.run.smithi113.stdout:NAME="Ubuntu" 2023-12-02T23:22:29.562 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_ID="22.04" 2023-12-02T23:22:29.562 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-12-02T23:22:29.562 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_CODENAME=jammy 2023-12-02T23:22:29.562 INFO:teuthology.orchestra.run.smithi113.stdout:ID=ubuntu 2023-12-02T23:22:29.562 INFO:teuthology.orchestra.run.smithi113.stdout:ID_LIKE=debian 2023-12-02T23:22:29.562 INFO:teuthology.orchestra.run.smithi113.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-02T23:22:29.563 INFO:teuthology.orchestra.run.smithi113.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-02T23:22:29.563 INFO:teuthology.orchestra.run.smithi113.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-02T23:22:29.563 INFO:teuthology.orchestra.run.smithi113.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-02T23:22:29.563 INFO:teuthology.orchestra.run.smithi113.stdout:UBUNTU_CODENAME=jammy 2023-12-02T23:22:29.563 INFO:teuthology.lock.ops:Updating smithi113.front.sepia.ceph.com on lock server 2023-12-02T23:22:29.588 INFO:teuthology.lock.ops:Updating [smithi156.front.sepia.ceph.com]: set os type and version on server 2023-12-02T23:22:29.588 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-12-02T23:22:29.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-02T23:22:29.665 DEBUG:teuthology.orchestra.run.smithi156:> true 2023-12-02T23:22:29.886 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-12-02T23:22:29.886 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2023-12-02T23:22:29.934 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2023-12-02T23:22:29.934 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2023-12-02T23:22:29.983 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-12-02T23:22:29.983 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="Ubuntu" 2023-12-02T23:22:29.983 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="22.04" 2023-12-02T23:22:29.983 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-12-02T23:22:29.983 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_CODENAME=jammy 2023-12-02T23:22:29.983 INFO:teuthology.orchestra.run.smithi156.stdout:ID=ubuntu 2023-12-02T23:22:29.983 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE=debian 2023-12-02T23:22:29.983 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-02T23:22:29.983 INFO:teuthology.orchestra.run.smithi156.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-02T23:22:29.983 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-02T23:22:29.983 INFO:teuthology.orchestra.run.smithi156.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-02T23:22:29.983 INFO:teuthology.orchestra.run.smithi156.stdout:UBUNTU_CODENAME=jammy 2023-12-02T23:22:29.984 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2023-12-02T23:22:30.005 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-02T23:22:30.101 INFO:teuthology.dispatcher.supervisor:Running job 7475911 2023-12-02T23:22:30.101 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/bin/teuthology -v --owner scheduled_dis@teuthology --archive /home/teuthworker/archive/dis-2023-12-02_21:54:05-rbd-wip-dis-testing-distro-default-smithi/7475911 --name dis-2023-12-02_21:54:05-rbd-wip-dis-testing-distro-default-smithi --description rbd/migration/{1-base/install 2-clusters/{fixed-3 openstack} 3-objectstore/bluestore-low-osd-mem-target 4-supported-random-distro$/{ubuntu_latest} 5-data-pool/none 6-prepare/qcow2-http 7-io-workloads/qemu_xfstests 8-migrate-workloads/execute 9-cleanup/cleanup conf/{disable-pool-app}} -- /home/teuthworker/archive/dis-2023-12-02_21:54:05-rbd-wip-dis-testing-distro-default-smithi/7475911/orig.config.yaml 2023-12-02T23:22:30.104 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/dis-2023-12-02_21:54:05-rbd-wip-dis-testing-distro-default-smithi/7475911 2023-12-02T23:22:30.104 INFO:teuthology.dispatcher.supervisor:Job PID: 3495620 2023-12-02T23:22:30.104 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-02T23:24:30.204 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-02T23:26:30.371 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-02T23:28:30.540 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-02T23:30:30.711 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-02T23:32:30.879 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-02T23:34:31.051 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-02T23:36:31.215 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-02T23:38:31.384 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-02T23:40:31.550 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-02T23:40:31.616 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-02T23:40:31.731 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-02T23:40:31.731 INFO:teuthology.nuke:Checking targets against current locks 2023-12-02T23:40:31.780 DEBUG:teuthology.nuke:shortname: smithi072 2023-12-02T23:40:31.780 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-02T23:40:31.800 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2023-12-02_21:54:05-rbd-wip-dis-testing-distro-default-smithi/7475911', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-02 23:15:48.247365', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH8gWsvCcwEYztFWaXF6W8XXfhvNsK2KPq6Yktajh/Id8kgzol6AepMEbznwipli1AX5jIlO7qB1OiKp0T4duxc='} 2023-12-02T23:40:31.804 DEBUG:teuthology.nuke:shortname: smithi113 2023-12-02T23:40:31.805 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-02T23:40:31.819 DEBUG:teuthology.nuke:shortname: smithi156 2023-12-02T23:40:31.819 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-02T23:40:31.825 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2023-12-02_21:54:05-rbd-wip-dis-testing-distro-default-smithi/7475911', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-02 23:15:48.248885', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPdgJunx7wZv5EW4+GNr2E5Vbqwg/LI4TGfya7GCwKd7POB79H/fnEmwfR374TNyNyCsIaf/yVirdB6YmO6NgKU='} 2023-12-02T23:40:31.842 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2023-12-02_21:54:05-rbd-wip-dis-testing-distro-default-smithi/7475911', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-02 23:15:48.250206', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCaUk7VnGsHycyTFkidwMyCpiDe3mqcpHmJ7Q2TbsWBMp6BTfja3WFd+g3clbJzvpsaubDuJRHZCG3k9pxNr92A='} 2023-12-02T23:40:31.852 INFO:teuthology.orchestra.console.smithi072:Power off 2023-12-02T23:40:31.853 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-02T23:40:31.876 DEBUG:teuthology.orchestra.console.smithi072:power off output: Chassis Power Control: Down/Off 2023-12-02T23:40:31.876 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T23:40:31.892 INFO:teuthology.orchestra.console.smithi156:Power off 2023-12-02T23:40:31.892 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-02T23:40:31.903 INFO:teuthology.orchestra.console.smithi113:Power off 2023-12-02T23:40:31.903 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-02T23:40:31.910 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off 2023-12-02T23:40:31.912 DEBUG:teuthology.orchestra.console.smithi156:power off output: Chassis Power Control: Down/Off 2023-12-02T23:40:31.912 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T23:40:31.922 DEBUG:teuthology.orchestra.console.smithi113:power off output: Chassis Power Control: Down/Off 2023-12-02T23:40:31.923 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-02T23:40:31.933 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is off 2023-12-02T23:40:31.946 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off 2023-12-02T23:40:32.012 INFO:teuthology.orchestra.console.smithi072:Power off completed 2023-12-02T23:40:32.034 INFO:teuthology.orchestra.console.smithi156:Power off completed 2023-12-02T23:40:32.046 INFO:teuthology.orchestra.console.smithi113:Power off completed 2023-12-02T23:40:32.166 INFO:teuthology.lock.ops:unlocked: smithi072.front.sepia.ceph.com 2023-12-02T23:40:32.189 INFO:teuthology.lock.ops:unlocked: smithi113.front.sepia.ceph.com 2023-12-02T23:40:32.198 INFO:teuthology.lock.ops:unlocked: smithi156.front.sepia.ceph.com