2023-10-20T14:00:13.627 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-20T14:00:13.629 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:00:13.964 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6ca10a8e50>, signals=[15]) 2023-10-20T14:00:13.965 INFO:teuthology.lock.ops:Start node 'smithi003.front.sepia.ceph.com' reimaging 2023-10-20T14:00:13.966 INFO:teuthology.lock.ops:Updating [smithi003.front.sepia.ceph.com]: reset os type and version on server 2023-10-20T14:00:13.966 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2023-10-20T14:00:13.989 INFO:teuthology.lock.ops:Start node 'smithi016.front.sepia.ceph.com' reimaging 2023-10-20T14:00:13.989 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: reset os type and version on server 2023-10-20T14:00:13.989 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-10-20T14:00:13.991 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /host {"name": "smithi003"} 2023-10-20T14:00:14.013 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /host {"name": "smithi016"} 2023-10-20T14:00:14.079 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /image {"name": "smithi_ubuntu_20.04"} 2023-10-20T14:00:14.082 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /image {"name": "smithi_ubuntu_20.04"} 2023-10-20T14:00:14.124 DEBUG:teuthology.provision.fog.smithi003:Requesting image smithi_ubuntu_20.04 (ID 2) 2023-10-20T14:00:14.124 DEBUG:teuthology.provision.fog.smithi003:FOG request: PUT /host/109 {"imageID": 2} 2023-10-20T14:00:14.127 DEBUG:teuthology.provision.fog.smithi016:Requesting image smithi_ubuntu_20.04 (ID 2) 2023-10-20T14:00:14.128 DEBUG:teuthology.provision.fog.smithi016:FOG request: PUT /host/122 {"imageID": 2} 2023-10-20T14:00:14.168 INFO:teuthology.provision.fog.smithi003:Scheduling deploy of ubuntu 20.04 2023-10-20T14:00:14.169 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-20T14:00:14.180 INFO:teuthology.provision.fog.smithi016:Scheduling deploy of ubuntu 20.04 2023-10-20T14:00:14.180 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:00:14.219 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /tasktype {"name": "deploy"} 2023-10-20T14:00:14.225 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /tasktype {"name": "deploy"} 2023-10-20T14:00:14.256 DEBUG:teuthology.provision.fog.smithi016:FOG request: POST /host/122/task {"taskTypeID": 1} 2023-10-20T14:00:14.260 DEBUG:teuthology.provision.fog.smithi003:FOG request: POST /host/109/task {"taskTypeID": 1} 2023-10-20T14:00:14.309 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:00:14.327 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-20T14:00:14.387 INFO:teuthology.orchestra.console.smithi016:Power off 2023-10-20T14:00:14.388 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-20T14:00:14.400 INFO:teuthology.orchestra.console.smithi003:Power off 2023-10-20T14:00:14.401 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-20T14:00:14.409 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2023-10-20T14:00:14.409 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-20T14:00:14.421 DEBUG:teuthology.orchestra.console.smithi003:power off output: Chassis Power Control: Down/Off 2023-10-20T14:00:14.422 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-20T14:00:14.430 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-10-20T14:00:14.444 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off 2023-10-20T14:00:14.532 INFO:teuthology.orchestra.console.smithi016:Power off completed 2023-10-20T14:00:14.545 INFO:teuthology.orchestra.console.smithi003:Power off completed 2023-10-20T14:00:14.633 INFO:teuthology.orchestra.console.smithi016:Power on 2023-10-20T14:00:14.634 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-20T14:00:14.646 INFO:teuthology.orchestra.console.smithi003:Power on 2023-10-20T14:00:14.647 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-20T14:00:14.656 DEBUG:teuthology.orchestra.console.smithi016:power on output: Chassis Power Control: Up/On 2023-10-20T14:00:14.657 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-20T14:00:14.667 DEBUG:teuthology.orchestra.console.smithi003:power on output: Chassis Power Control: Up/On 2023-10-20T14:00:14.668 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-20T14:00:14.680 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-10-20T14:00:14.687 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off 2023-10-20T14:00:18.682 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-20T14:00:18.689 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-20T14:00:18.791 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2023-10-20T14:00:18.798 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2023-10-20T14:00:18.893 INFO:teuthology.orchestra.console.smithi016:Power on completed 2023-10-20T14:00:18.900 INFO:teuthology.orchestra.console.smithi003:Power on completed 2023-10-20T14:00:18.994 INFO:teuthology.provision.fog.smithi016:Waiting for deploy to finish 2023-10-20T14:00:18.995 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:00:19.000 INFO:teuthology.provision.fog.smithi003:Waiting for deploy to finish 2023-10-20T14:00:19.001 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-20T14:00:34.068 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:00:34.070 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-20T14:00:49.140 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-20T14:00:49.142 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:01:04.218 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-20T14:01:04.220 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:01:19.294 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-20T14:01:19.296 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:01:34.366 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-20T14:01:34.368 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:01:49.434 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:01:49.436 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-20T14:02:04.518 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-20T14:02:04.520 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:02:19.594 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-20T14:02:19.597 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:02:34.678 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-20T14:02:34.680 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:02:49.754 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:02:49.767 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-20T14:03:04.862 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:03:04.864 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-20T14:03:19.938 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-20T14:03:19.939 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:03:35.010 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-20T14:03:35.012 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:03:50.082 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:03:50.084 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-10-20T14:03:50.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-20T14:03:50.270 ERROR:teuthology.orchestra.connection:Error authenticating with smithi003.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-10-20T14:04:04.602 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-10-20T14:04:05.150 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None 2023-10-20T14:04:05.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-20T14:04:10.604 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-20T14:04:13.658 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-10-20T14:04:19.660 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-20T14:04:22.718 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-10-20T14:04:23.546 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-10-20T14:04:28.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-20T14:04:29.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-20T14:04:31.774 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-10-20T14:04:32.602 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-10-20T14:04:37.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-20T14:04:38.604 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-20T14:04:40.830 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-10-20T14:04:41.662 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-10-20T14:04:46.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-20T14:04:47.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-20T14:05:46.834 WARNING:teuthology.provision.fog.smithi003:timed out 2023-10-20T14:05:47.666 WARNING:teuthology.provision.fog.smithi016:timed out 2023-10-20T14:05:52.837 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-20T14:05:53.047 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-20T14:05:53.048 DEBUG:teuthology.orchestra.run.smithi003:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-20T14:05:53.600 INFO:teuthology.provision.fog.smithi003:Node is ready 2023-10-20T14:05:53.601 DEBUG:teuthology.orchestra.run.smithi003:> hostname 2023-10-20T14:05:53.607 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003.front.sepia.ceph.com 2023-10-20T14:05:53.607 DEBUG:teuthology.orchestra.run.smithi003:> grep smithi003.front.sepia.ceph.com /etc/hosts 2023-10-20T14:05:53.656 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3 smithi003.front.sepia.ceph.com smithi003 2023-10-20T14:05:53.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname smithi003 2023-10-20T14:05:53.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-20T14:05:53.744 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/smithi003.front.sepia.ceph.com/smithi003/g' /etc/hosts 2023-10-20T14:05:53.761 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/172.21.15.3/172.21.15.3/g' /etc/hosts 2023-10-20T14:05:53.774 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/smithi003.front.sepia.ceph.com/smithi003/g' /etc/hostname 2023-10-20T14:05:53.834 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname smithi003 2023-10-20T14:05:53.843 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2023-10-20T14:05:53.849 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="Ubuntu" 2023-10-20T14:05:53.850 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-20T14:05:53.850 INFO:teuthology.orchestra.run.smithi003.stdout:ID=ubuntu 2023-10-20T14:05:53.850 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE=debian 2023-10-20T14:05:53.850 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-20T14:05:53.851 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="20.04" 2023-10-20T14:05:53.851 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-20T14:05:53.851 INFO:teuthology.orchestra.run.smithi003.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-20T14:05:53.851 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-20T14:05:53.851 INFO:teuthology.orchestra.run.smithi003.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-20T14:05:53.852 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_CODENAME=focal 2023-10-20T14:05:53.852 INFO:teuthology.orchestra.run.smithi003.stdout:UBUNTU_CODENAME=focal 2023-10-20T14:05:53.852 INFO:teuthology.provision.fog.smithi003:Deploy complete! 2023-10-20T14:05:53.854 DEBUG:teuthology.parallel:result is None 2023-10-20T14:05:53.864 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-20T14:05:53.864 DEBUG:teuthology.orchestra.run.smithi016:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-20T14:05:54.405 INFO:teuthology.provision.fog.smithi016:Node is ready 2023-10-20T14:05:54.405 DEBUG:teuthology.orchestra.run.smithi016:> hostname 2023-10-20T14:05:54.412 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016.front.sepia.ceph.com 2023-10-20T14:05:54.412 DEBUG:teuthology.orchestra.run.smithi016:> grep smithi016.front.sepia.ceph.com /etc/hosts 2023-10-20T14:05:54.420 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16 smithi016.front.sepia.ceph.com smithi016 2023-10-20T14:05:54.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname smithi016 2023-10-20T14:05:54.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/smithi016.front.sepia.ceph.com/smithi016/g' /etc/hosts 2023-10-20T14:05:54.579 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/172.21.15.16/172.21.15.16/g' /etc/hosts 2023-10-20T14:05:54.590 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/smithi016.front.sepia.ceph.com/smithi016/g' /etc/hostname 2023-10-20T14:05:54.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname smithi016 2023-10-20T14:05:54.661 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2023-10-20T14:05:54.667 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Ubuntu" 2023-10-20T14:05:54.667 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-20T14:05:54.667 INFO:teuthology.orchestra.run.smithi016.stdout:ID=ubuntu 2023-10-20T14:05:54.668 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE=debian 2023-10-20T14:05:54.668 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-20T14:05:54.668 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="20.04" 2023-10-20T14:05:54.668 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-20T14:05:54.668 INFO:teuthology.orchestra.run.smithi016.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-20T14:05:54.669 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-20T14:05:54.669 INFO:teuthology.orchestra.run.smithi016.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-20T14:05:54.669 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_CODENAME=focal 2023-10-20T14:05:54.669 INFO:teuthology.orchestra.run.smithi016.stdout:UBUNTU_CODENAME=focal 2023-10-20T14:05:54.670 INFO:teuthology.provision.fog.smithi016:Deploy complete! 2023-10-20T14:05:54.670 DEBUG:teuthology.parallel:result is None 2023-10-20T14:05:54.921 INFO:teuthology.lock.ops:Checking smithi003.front.sepia.ceph.com 2023-10-20T14:05:54.922 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-20T14:05:54.946 INFO:teuthology.lock.ops:Checking smithi016.front.sepia.ceph.com 2023-10-20T14:05:54.947 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-20T14:05:54.965 INFO:teuthology.lock.ops:Updating [smithi003.front.sepia.ceph.com]: set os type and version on server 2023-10-20T14:05:54.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-10-20T14:05:54.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-20T14:05:55.016 DEBUG:teuthology.orchestra.run.smithi003:> true 2023-10-20T14:05:55.143 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-10-20T14:05:55.144 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2023-10-20T14:05:55.150 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2023-10-20T14:05:55.151 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2023-10-20T14:05:55.157 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="Ubuntu" 2023-10-20T14:05:55.157 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-20T14:05:55.157 INFO:teuthology.orchestra.run.smithi003.stdout:ID=ubuntu 2023-10-20T14:05:55.157 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE=debian 2023-10-20T14:05:55.158 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-20T14:05:55.158 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="20.04" 2023-10-20T14:05:55.158 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-20T14:05:55.158 INFO:teuthology.orchestra.run.smithi003.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-20T14:05:55.158 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-20T14:05:55.159 INFO:teuthology.orchestra.run.smithi003.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-20T14:05:55.159 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_CODENAME=focal 2023-10-20T14:05:55.159 INFO:teuthology.orchestra.run.smithi003.stdout:UBUNTU_CODENAME=focal 2023-10-20T14:05:55.159 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2023-10-20T14:05:55.180 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: set os type and version on server 2023-10-20T14:05:55.181 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-10-20T14:05:55.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-20T14:05:55.232 DEBUG:teuthology.orchestra.run.smithi016:> true 2023-10-20T14:05:55.360 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-10-20T14:05:55.360 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2023-10-20T14:05:55.367 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2023-10-20T14:05:55.367 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2023-10-20T14:05:55.373 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Ubuntu" 2023-10-20T14:05:55.374 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-20T14:05:55.374 INFO:teuthology.orchestra.run.smithi016.stdout:ID=ubuntu 2023-10-20T14:05:55.374 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE=debian 2023-10-20T14:05:55.374 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-20T14:05:55.374 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="20.04" 2023-10-20T14:05:55.375 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-20T14:05:55.375 INFO:teuthology.orchestra.run.smithi016.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-20T14:05:55.375 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-20T14:05:55.375 INFO:teuthology.orchestra.run.smithi016.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-20T14:05:55.375 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_CODENAME=focal 2023-10-20T14:05:55.376 INFO:teuthology.orchestra.run.smithi016.stdout:UBUNTU_CODENAME=focal 2023-10-20T14:05:55.376 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-10-20T14:05:55.396 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:05:55.505 INFO:teuthology.dispatcher.supervisor:Running job 7427626 2023-10-20T14:05:55.505 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2023-10-14_14:23:11-upgrade:pacific-x-reef-distro-default-smithi/7427626 --name teuthology-2023-10-14_14:23:11-upgrade:pacific-x-reef-distro-default-smithi --description upgrade:pacific-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} -- /home/teuthworker/archive/teuthology-2023-10-14_14:23:11-upgrade:pacific-x-reef-distro-default-smithi/7427626/orig.config.yaml 2023-10-20T14:05:55.509 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2023-10-14_14:23:11-upgrade:pacific-x-reef-distro-default-smithi/7427626 2023-10-20T14:05:55.510 INFO:teuthology.dispatcher.supervisor:Job PID: 2182725 2023-10-20T14:05:55.510 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-10-20T14:07:55.512 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:09:55.634 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:11:55.705 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:13:55.770 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:15:55.843 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:17:55.951 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:19:56.015 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:21:56.079 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:23:56.148 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:25:56.211 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:27:56.276 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:29:56.340 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:31:56.412 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:33:56.486 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:35:56.550 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:37:56.640 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:39:56.708 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:41:56.778 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:43:56.847 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:45:56.908 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:47:56.981 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:49:57.044 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:51:57.107 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:53:57.175 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:55:57.246 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:57:57.310 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T14:59:57.374 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:01:57.438 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:03:57.504 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:05:57.603 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:07:57.665 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:09:57.736 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:11:57.801 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:13:57.869 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:15:57.943 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:17:58.011 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:19:58.082 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:21:58.153 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:23:58.219 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:25:58.293 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:27:58.392 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:29:58.469 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:31:58.539 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:33:58.614 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:35:58.688 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:37:58.757 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:39:58.822 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:41:58.894 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:43:58.997 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-20T15:43:59.067 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-10-20T15:43:59.262 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-10-20T15:43:59.262 INFO:teuthology.nuke:Checking targets against current locks 2023-10-20T15:43:59.312 DEBUG:teuthology.nuke:shortname: smithi003 2023-10-20T15:43:59.313 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-20T15:43:59.340 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2023-10-14_14:23:11-upgrade:pacific-x-reef-distro-default-smithi/7427626', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-20 14:00:12.756671', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-20T15:43:59.344 DEBUG:teuthology.nuke:shortname: smithi016 2023-10-20T15:43:59.344 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-20T15:43:59.369 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2023-10-14_14:23:11-upgrade:pacific-x-reef-distro-default-smithi/7427626', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-20 14:00:12.758134', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-20T15:43:59.392 INFO:teuthology.orchestra.console.smithi003:Power off 2023-10-20T15:43:59.393 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-20T15:43:59.418 DEBUG:teuthology.orchestra.console.smithi003:power off output: Chassis Power Control: Down/Off 2023-10-20T15:43:59.419 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-20T15:43:59.434 INFO:teuthology.orchestra.console.smithi016:Power off 2023-10-20T15:43:59.435 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-20T15:43:59.443 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off 2023-10-20T15:43:59.457 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2023-10-20T15:43:59.457 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-20T15:43:59.487 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-10-20T15:43:59.544 INFO:teuthology.orchestra.console.smithi003:Power off completed 2023-10-20T15:43:59.588 INFO:teuthology.orchestra.console.smithi016:Power off completed 2023-10-20T15:43:59.709 INFO:teuthology.lock.ops:unlocked: smithi003.front.sepia.ceph.com 2023-10-20T15:43:59.742 INFO:teuthology.lock.ops:unlocked: smithi016.front.sepia.ceph.com