2024-01-13T04:44:29.544 INFO:root:teuthology version: 0.0.1.dev249+gcd45576
2024-01-13T04:44:29.545 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-13T04:44:29.723 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fa3af330f40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fa3b0396c10>, signals=[15])
2024-01-13T04:44:29.723 INFO:teuthology.lock.ops:Start node 'smithi081.front.sepia.ceph.com' reimaging
2024-01-13T04:44:29.724 INFO:teuthology.lock.ops:Updating [smithi081.front.sepia.ceph.com]: reset os type and version on server
2024-01-13T04:44:29.724 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server
2024-01-13T04:44:29.745 INFO:teuthology.lock.ops:Start node 'smithi119.front.sepia.ceph.com' reimaging
2024-01-13T04:44:29.746 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: reset os type and version on server
2024-01-13T04:44:29.746 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server
2024-01-13T04:44:29.770 INFO:teuthology.lock.ops:Start node 'smithi125.front.sepia.ceph.com' reimaging
2024-01-13T04:44:29.770 INFO:teuthology.lock.ops:Updating [smithi125.front.sepia.ceph.com]: reset os type and version on server
2024-01-13T04:44:29.770 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server
2024-01-13T04:44:29.867 DEBUG:teuthology.provision.fog.smithi081:Requesting image smithi_ubuntu_20.04 (ID 2)
2024-01-13T04:44:29.880 DEBUG:teuthology.provision.fog.smithi119:Requesting image smithi_ubuntu_20.04 (ID 2)
2024-01-13T04:44:29.906 DEBUG:teuthology.provision.fog.smithi125:Requesting image smithi_ubuntu_20.04 (ID 2)
2024-01-13T04:44:29.909 INFO:teuthology.provision.fog.smithi081:Scheduling deploy of ubuntu 20.04
2024-01-13T04:44:29.922 INFO:teuthology.provision.fog.smithi119:Scheduling deploy of ubuntu 20.04
2024-01-13T04:44:29.954 INFO:teuthology.provision.fog.smithi125:Scheduling deploy of ubuntu 20.04
2024-01-13T04:44:30.182 INFO:teuthology.orchestra.console.smithi119:Power off
2024-01-13T04:44:30.182 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-13T04:44:30.192 INFO:teuthology.orchestra.console.smithi081:Power off
2024-01-13T04:44:30.193 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-13T04:44:30.204 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off
2024-01-13T04:44:30.204 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-13T04:44:30.213 DEBUG:teuthology.orchestra.console.smithi081:power off output: Chassis Power Control: Down/Off
2024-01-13T04:44:30.214 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-13T04:44:30.227 INFO:teuthology.orchestra.console.smithi125:Power off
2024-01-13T04:44:30.227 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-13T04:44:30.236 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is off
2024-01-13T04:44:30.236 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on
2024-01-13T04:44:30.248 DEBUG:teuthology.orchestra.console.smithi125:power off output: Chassis Power Control: Down/Off
2024-01-13T04:44:30.248 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-13T04:44:30.270 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off
2024-01-13T04:44:30.337 INFO:teuthology.orchestra.console.smithi081:Power off completed
2024-01-13T04:44:30.371 INFO:teuthology.orchestra.console.smithi125:Power off completed
2024-01-13T04:44:30.438 INFO:teuthology.orchestra.console.smithi081:Power on
2024-01-13T04:44:30.439 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-13T04:44:30.460 DEBUG:teuthology.orchestra.console.smithi081:power on output: Chassis Power Control: Up/On
2024-01-13T04:44:30.460 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-13T04:44:30.472 INFO:teuthology.orchestra.console.smithi125:Power on
2024-01-13T04:44:30.472 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-13T04:44:30.483 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is off
2024-01-13T04:44:30.493 DEBUG:teuthology.orchestra.console.smithi125:power on output: Chassis Power Control: Up/On
2024-01-13T04:44:30.493 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-13T04:44:30.514 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off
2024-01-13T04:44:34.241 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-13T04:44:34.350 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on
2024-01-13T04:44:34.484 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-13T04:44:34.515 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-13T04:44:34.592 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on
2024-01-13T04:44:34.623 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off
2024-01-13T04:44:34.694 INFO:teuthology.orchestra.console.smithi081:Power on completed
2024-01-13T04:44:34.794 INFO:teuthology.provision.fog.smithi081:Waiting for deploy to finish
2024-01-13T04:44:38.353 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-13T04:44:38.463 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off
2024-01-13T04:44:38.563 INFO:teuthology.orchestra.console.smithi119:Power off completed
2024-01-13T04:44:38.624 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-13T04:44:38.664 INFO:teuthology.orchestra.console.smithi119:Power on
2024-01-13T04:44:38.665 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-13T04:44:38.688 DEBUG:teuthology.orchestra.console.smithi119:power on output: Chassis Power Control: Up/On
2024-01-13T04:44:38.688 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-13T04:44:38.711 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off
2024-01-13T04:44:38.741 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on
2024-01-13T04:44:38.842 INFO:teuthology.orchestra.console.smithi125:Power on completed
2024-01-13T04:44:38.942 INFO:teuthology.provision.fog.smithi125:Waiting for deploy to finish
2024-01-13T04:44:42.716 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-13T04:44:42.825 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off
2024-01-13T04:44:46.828 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-13T04:44:46.936 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on
2024-01-13T04:44:47.037 INFO:teuthology.orchestra.console.smithi119:Power on completed
2024-01-13T04:44:47.138 INFO:teuthology.provision.fog.smithi119:Waiting for deploy to finish
2024-01-13T04:48:10.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:48:21.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:48:24.156 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2024-01-13T04:48:30.158 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:48:33.212 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2024-01-13T04:48:33.486 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:48:36.540 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119
2024-01-13T04:48:39.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:48:42.268 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2024-01-13T04:48:42.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:48:45.596 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119
2024-01-13T04:48:48.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:48:51.328 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2024-01-13T04:48:51.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:48:54.652 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119
2024-01-13T04:48:57.329 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:49:00.380 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2024-01-13T04:49:00.653 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:49:03.708 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119
2024-01-13T04:49:06.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:49:09.436 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81
2024-01-13T04:49:09.709 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:49:10.250 ERROR:teuthology.orchestra.connection:Error authenticating with smithi125.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/connection.py", line 106, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 457, in connect
    server_key = t.get_remote_server_key()
  File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 918, in get_remote_server_key
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2024-01-13T04:49:12.764 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119
2024-01-13T04:49:15.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:49:18.765 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:49:21.820 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119
2024-01-13T04:49:27.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:49:47.534 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-13T04:49:47.535 DEBUG:teuthology.orchestra.run.smithi081:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-13T04:50:09.053 INFO:teuthology.provision.fog.smithi081:Node is ready
2024-01-13T04:50:09.053 DEBUG:teuthology.orchestra.run.smithi081:> hostname
2024-01-13T04:50:09.060 INFO:teuthology.orchestra.run.smithi081.stdout:smithi081.front.sepia.ceph.com
2024-01-13T04:50:09.060 DEBUG:teuthology.orchestra.run.smithi081:> grep smithi081.front.sepia.ceph.com /etc/hosts
2024-01-13T04:50:09.108 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81 smithi081.front.sepia.ceph.com smithi081
2024-01-13T04:50:09.108 DEBUG:teuthology.orchestra.run.smithi081:> sudo hostname smithi081
2024-01-13T04:50:09.189 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i -e 's/smithi081.front.sepia.ceph.com/smithi081/g' /etc/hosts
2024-01-13T04:50:09.206 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i -e 's/172.21.15.81/172.21.15.81/g' /etc/hosts
2024-01-13T04:50:09.223 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i -e 's/smithi081.front.sepia.ceph.com/smithi081/g' /etc/hostname
2024-01-13T04:50:09.284 DEBUG:teuthology.orchestra.run.smithi081:> sudo hostname smithi081
2024-01-13T04:50:09.293 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release
2024-01-13T04:50:09.298 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="Ubuntu"
2024-01-13T04:50:09.298 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="20.04.4 LTS (Focal Fossa)"
2024-01-13T04:50:09.298 INFO:teuthology.orchestra.run.smithi081.stdout:ID=ubuntu
2024-01-13T04:50:09.298 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE=debian
2024-01-13T04:50:09.298 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS"
2024-01-13T04:50:09.298 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="20.04"
2024-01-13T04:50:09.298 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://www.ubuntu.com/"
2024-01-13T04:50:09.298 INFO:teuthology.orchestra.run.smithi081.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-01-13T04:50:09.298 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-01-13T04:50:09.298 INFO:teuthology.orchestra.run.smithi081.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-01-13T04:50:09.298 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_CODENAME=focal
2024-01-13T04:50:09.298 INFO:teuthology.orchestra.run.smithi081.stdout:UBUNTU_CODENAME=focal
2024-01-13T04:50:09.299 INFO:teuthology.provision.fog.smithi081:Deploy complete!
2024-01-13T04:50:09.301 DEBUG:teuthology.parallel:result is None
2024-01-13T04:50:14.256 WARNING:teuthology.provision.fog.smithi125:timed out
2024-01-13T04:50:20.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:50:20.525 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-13T04:50:20.525 DEBUG:teuthology.orchestra.run.smithi125:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-13T04:50:21.007 INFO:teuthology.provision.fog.smithi125:Node is ready
2024-01-13T04:50:21.008 DEBUG:teuthology.orchestra.run.smithi125:> hostname
2024-01-13T04:50:21.014 INFO:teuthology.orchestra.run.smithi125.stdout:smithi125.front.sepia.ceph.com
2024-01-13T04:50:21.014 DEBUG:teuthology.orchestra.run.smithi125:> grep smithi125.front.sepia.ceph.com /etc/hosts
2024-01-13T04:50:21.063 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.125 smithi125.front.sepia.ceph.com smithi125
2024-01-13T04:50:21.064 DEBUG:teuthology.orchestra.run.smithi125:> sudo hostname smithi125
2024-01-13T04:50:21.142 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/smithi125.front.sepia.ceph.com/smithi125/g' /etc/hosts
2024-01-13T04:50:21.159 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/172.21.15.125/172.21.15.125/g' /etc/hosts
2024-01-13T04:50:21.215 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/smithi125.front.sepia.ceph.com/smithi125/g' /etc/hostname
2024-01-13T04:50:21.230 DEBUG:teuthology.orchestra.run.smithi125:> sudo hostname smithi125
2024-01-13T04:50:21.239 DEBUG:teuthology.orchestra.run.smithi125:> cat /etc/os-release
2024-01-13T04:50:21.243 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="Ubuntu"
2024-01-13T04:50:21.244 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="20.04.4 LTS (Focal Fossa)"
2024-01-13T04:50:21.244 INFO:teuthology.orchestra.run.smithi125.stdout:ID=ubuntu
2024-01-13T04:50:21.244 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE=debian
2024-01-13T04:50:21.244 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS"
2024-01-13T04:50:21.244 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="20.04"
2024-01-13T04:50:21.244 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://www.ubuntu.com/"
2024-01-13T04:50:21.244 INFO:teuthology.orchestra.run.smithi125.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-01-13T04:50:21.244 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-01-13T04:50:21.244 INFO:teuthology.orchestra.run.smithi125.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-01-13T04:50:21.244 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_CODENAME=focal
2024-01-13T04:50:21.244 INFO:teuthology.orchestra.run.smithi125.stdout:UBUNTU_CODENAME=focal
2024-01-13T04:50:21.245 INFO:teuthology.provision.fog.smithi125:Deploy complete!
2024-01-13T04:50:21.245 DEBUG:teuthology.parallel:result is None
2024-01-13T04:50:27.824 WARNING:teuthology.provision.fog.smithi119:timed out
2024-01-13T04:50:33.826 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:50:34.074 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-13T04:50:34.075 DEBUG:teuthology.orchestra.run.smithi119:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-13T04:50:34.598 INFO:teuthology.provision.fog.smithi119:Node is ready
2024-01-13T04:50:34.599 DEBUG:teuthology.orchestra.run.smithi119:> hostname
2024-01-13T04:50:34.605 INFO:teuthology.orchestra.run.smithi119.stdout:smithi119.front.sepia.ceph.com
2024-01-13T04:50:34.605 DEBUG:teuthology.orchestra.run.smithi119:> grep smithi119.front.sepia.ceph.com /etc/hosts
2024-01-13T04:50:34.657 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119 smithi119.front.sepia.ceph.com smithi119
2024-01-13T04:50:34.657 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname smithi119
2024-01-13T04:50:34.736 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i -e 's/smithi119.front.sepia.ceph.com/smithi119/g' /etc/hosts
2024-01-13T04:50:34.750 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i -e 's/172.21.15.119/172.21.15.119/g' /etc/hosts
2024-01-13T04:50:34.810 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i -e 's/smithi119.front.sepia.ceph.com/smithi119/g' /etc/hostname
2024-01-13T04:50:34.826 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname smithi119
2024-01-13T04:50:34.840 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release
2024-01-13T04:50:34.844 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Ubuntu"
2024-01-13T04:50:34.844 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="20.04.4 LTS (Focal Fossa)"
2024-01-13T04:50:34.844 INFO:teuthology.orchestra.run.smithi119.stdout:ID=ubuntu
2024-01-13T04:50:34.845 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE=debian
2024-01-13T04:50:34.845 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS"
2024-01-13T04:50:34.845 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="20.04"
2024-01-13T04:50:34.845 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.ubuntu.com/"
2024-01-13T04:50:34.845 INFO:teuthology.orchestra.run.smithi119.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-01-13T04:50:34.845 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-01-13T04:50:34.845 INFO:teuthology.orchestra.run.smithi119.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-01-13T04:50:34.845 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_CODENAME=focal
2024-01-13T04:50:34.845 INFO:teuthology.orchestra.run.smithi119.stdout:UBUNTU_CODENAME=focal
2024-01-13T04:50:34.845 INFO:teuthology.provision.fog.smithi119:Deploy complete!
2024-01-13T04:50:34.846 DEBUG:teuthology.parallel:result is None
2024-01-13T04:50:35.146 INFO:teuthology.lock.ops:Checking smithi081.front.sepia.ceph.com
2024-01-13T04:50:35.146 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-13T04:50:35.173 INFO:teuthology.lock.ops:Checking smithi119.front.sepia.ceph.com
2024-01-13T04:50:35.173 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-13T04:50:35.197 INFO:teuthology.lock.ops:Checking smithi125.front.sepia.ceph.com
2024-01-13T04:50:35.197 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-13T04:50:35.215 INFO:teuthology.lock.ops:Updating [smithi081.front.sepia.ceph.com]: set os type and version on server
2024-01-13T04:50:35.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com'
2024-01-13T04:50:35.216 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:50:35.307 DEBUG:teuthology.orchestra.run.smithi081:> true
2024-01-13T04:50:35.513 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com'
2024-01-13T04:50:35.514 DEBUG:teuthology.orchestra.run.smithi081:> uname -m
2024-01-13T04:50:35.519 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64
2024-01-13T04:50:35.520 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release
2024-01-13T04:50:35.526 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="Ubuntu"
2024-01-13T04:50:35.526 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="20.04.4 LTS (Focal Fossa)"
2024-01-13T04:50:35.526 INFO:teuthology.orchestra.run.smithi081.stdout:ID=ubuntu
2024-01-13T04:50:35.526 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE=debian
2024-01-13T04:50:35.526 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS"
2024-01-13T04:50:35.526 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="20.04"
2024-01-13T04:50:35.526 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://www.ubuntu.com/"
2024-01-13T04:50:35.526 INFO:teuthology.orchestra.run.smithi081.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-01-13T04:50:35.526 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-01-13T04:50:35.526 INFO:teuthology.orchestra.run.smithi081.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-01-13T04:50:35.526 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_CODENAME=focal
2024-01-13T04:50:35.526 INFO:teuthology.orchestra.run.smithi081.stdout:UBUNTU_CODENAME=focal
2024-01-13T04:50:35.527 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server
2024-01-13T04:50:35.548 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: set os type and version on server
2024-01-13T04:50:35.548 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com'
2024-01-13T04:50:35.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:50:35.636 DEBUG:teuthology.orchestra.run.smithi119:> true
2024-01-13T04:50:35.745 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com'
2024-01-13T04:50:35.745 DEBUG:teuthology.orchestra.run.smithi119:> uname -m
2024-01-13T04:50:35.751 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64
2024-01-13T04:50:35.751 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release
2024-01-13T04:50:35.803 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Ubuntu"
2024-01-13T04:50:35.804 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="20.04.4 LTS (Focal Fossa)"
2024-01-13T04:50:35.804 INFO:teuthology.orchestra.run.smithi119.stdout:ID=ubuntu
2024-01-13T04:50:35.804 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE=debian
2024-01-13T04:50:35.804 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS"
2024-01-13T04:50:35.804 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="20.04"
2024-01-13T04:50:35.804 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.ubuntu.com/"
2024-01-13T04:50:35.804 INFO:teuthology.orchestra.run.smithi119.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-01-13T04:50:35.804 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-01-13T04:50:35.804 INFO:teuthology.orchestra.run.smithi119.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-01-13T04:50:35.804 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_CODENAME=focal
2024-01-13T04:50:35.804 INFO:teuthology.orchestra.run.smithi119.stdout:UBUNTU_CODENAME=focal
2024-01-13T04:50:35.805 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server
2024-01-13T04:50:35.831 INFO:teuthology.lock.ops:Updating [smithi125.front.sepia.ceph.com]: set os type and version on server
2024-01-13T04:50:35.831 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com'
2024-01-13T04:50:35.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-13T04:50:35.919 DEBUG:teuthology.orchestra.run.smithi125:> true
2024-01-13T04:50:36.122 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi125.front.sepia.ceph.com'
2024-01-13T04:50:36.122 DEBUG:teuthology.orchestra.run.smithi125:> uname -m
2024-01-13T04:50:36.129 INFO:teuthology.orchestra.run.smithi125.stdout:x86_64
2024-01-13T04:50:36.130 DEBUG:teuthology.orchestra.run.smithi125:> cat /etc/os-release
2024-01-13T04:50:36.136 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="Ubuntu"
2024-01-13T04:50:36.136 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="20.04.4 LTS (Focal Fossa)"
2024-01-13T04:50:36.136 INFO:teuthology.orchestra.run.smithi125.stdout:ID=ubuntu
2024-01-13T04:50:36.136 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE=debian
2024-01-13T04:50:36.136 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS"
2024-01-13T04:50:36.137 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="20.04"
2024-01-13T04:50:36.137 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://www.ubuntu.com/"
2024-01-13T04:50:36.137 INFO:teuthology.orchestra.run.smithi125.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-01-13T04:50:36.137 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-01-13T04:50:36.137 INFO:teuthology.orchestra.run.smithi125.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-01-13T04:50:36.137 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_CODENAME=focal
2024-01-13T04:50:36.138 INFO:teuthology.orchestra.run.smithi125.stdout:UBUNTU_CODENAME=focal
2024-01-13T04:50:36.138 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server
2024-01-13T04:50:36.165 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-13T04:50:36.268 INFO:teuthology.dispatcher.supervisor:Running job 7515223
2024-01-13T04:50:36.269 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-01-12_15:24:25-rados-wip-yuri5-testing-2024-01-11-1300-pacific-distro-default-smithi/7515223 --name yuriw-2024-01-12_15:24:25-rados-wip-yuri5-testing-2024-01-11-1300-pacific-distro-default-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} -- /home/teuthworker/archive/yuriw-2024-01-12_15:24:25-rados-wip-yuri5-testing-2024-01-11-1300-pacific-distro-default-smithi/7515223/orig.config.yaml
2024-01-13T04:50:36.272 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2024-01-12_15:24:25-rados-wip-yuri5-testing-2024-01-11-1300-pacific-distro-default-smithi/7515223
2024-01-13T04:50:36.272 INFO:teuthology.dispatcher.supervisor:Job PID: 2381620
2024-01-13T04:50:36.272 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-01-13T04:50:43.392 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2024-01-13T04:52:36.368 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-13T04:54:36.533 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-13T04:56:36.704 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-13T04:58:36.876 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-13T05:00:37.044 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-13T05:00:37.438 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2024-01-13T05:00:37.733 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2024-01-13T05:00:37.733 INFO:teuthology.nuke:Checking targets against current locks
2024-01-13T05:00:37.785 DEBUG:teuthology.nuke:shortname: smithi081
2024-01-13T05:00:37.785 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-13T05:00:37.807 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-12_15:24:25-rados-wip-yuri5-testing-2024-01-11-1300-pacific-distro-default-smithi/7515223', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-13 04:44:28.456784', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}
2024-01-13T05:00:37.810 DEBUG:teuthology.nuke:shortname: smithi119
2024-01-13T05:00:37.810 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-13T05:00:37.819 DEBUG:teuthology.nuke:shortname: smithi125
2024-01-13T05:00:37.820 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-13T05:00:37.829 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-12_15:24:25-rados-wip-yuri5-testing-2024-01-11-1300-pacific-distro-default-smithi/7515223', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-13 04:44:28.458348', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}
2024-01-13T05:00:37.840 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-12_15:24:25-rados-wip-yuri5-testing-2024-01-11-1300-pacific-distro-default-smithi/7515223', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-13 04:44:28.454938', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}
2024-01-13T05:00:37.860 INFO:teuthology.orchestra.console.smithi081:Power off
2024-01-13T05:00:37.860 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-13T05:00:37.880 INFO:teuthology.orchestra.console.smithi119:Power off
2024-01-13T05:00:37.880 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-13T05:00:37.890 DEBUG:teuthology.orchestra.console.smithi081:power off output: Chassis Power Control: Down/Off
2024-01-13T05:00:37.890 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-13T05:00:37.903 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off
2024-01-13T05:00:37.903 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-13T05:00:37.916 INFO:teuthology.orchestra.console.smithi125:Power off
2024-01-13T05:00:37.917 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-13T05:00:37.924 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is off
2024-01-13T05:00:37.925 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off
2024-01-13T05:00:37.937 DEBUG:teuthology.orchestra.console.smithi125:power off output: Chassis Power Control: Down/Off
2024-01-13T05:00:37.937 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-13T05:00:37.959 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off
2024-01-13T05:00:38.025 INFO:teuthology.orchestra.console.smithi081:Power off completed
2024-01-13T05:00:38.026 INFO:teuthology.orchestra.console.smithi119:Power off completed
2024-01-13T05:00:38.060 INFO:teuthology.orchestra.console.smithi125:Power off completed
2024-01-13T05:00:38.177 INFO:teuthology.lock.ops:unlocked: smithi119.front.sepia.ceph.com
2024-01-13T05:00:38.186 INFO:teuthology.lock.ops:unlocked: smithi081.front.sepia.ceph.com
2024-01-13T05:00:38.194 INFO:teuthology.lock.ops:unlocked: smithi125.front.sepia.ceph.com