2023-11-15T11:44:28.678 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-15T11:44:28.680 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T11:44:28.831 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc473453ca0>, signals=[15]) 2023-11-15T11:44:28.833 INFO:teuthology.lock.ops:Start node 'smithi162.front.sepia.ceph.com' reimaging 2023-11-15T11:44:28.834 INFO:teuthology.lock.ops:Updating [smithi162.front.sepia.ceph.com]: reset os type and version on server 2023-11-15T11:44:28.835 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2023-11-15T11:44:28.862 INFO:teuthology.lock.ops:Start node 'smithi187.front.sepia.ceph.com' reimaging 2023-11-15T11:44:28.863 INFO:teuthology.lock.ops:Updating [smithi187.front.sepia.ceph.com]: reset os type and version on server 2023-11-15T11:44:28.863 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2023-11-15T11:44:28.864 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /host {"name": "smithi162"} 2023-11-15T11:44:28.887 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /host {"name": "smithi187"} 2023-11-15T11:44:28.944 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /image {"name": "smithi_ubuntu_22.04"} 2023-11-15T11:44:28.954 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /image {"name": "smithi_ubuntu_22.04"} 2023-11-15T11:44:28.995 DEBUG:teuthology.provision.fog.smithi162:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-11-15T11:44:28.995 DEBUG:teuthology.provision.fog.smithi162:FOG request: PUT /host/267 {"imageID": 6} 2023-11-15T11:44:29.000 DEBUG:teuthology.provision.fog.smithi187:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-11-15T11:44:29.000 DEBUG:teuthology.provision.fog.smithi187:FOG request: PUT /host/292 {"imageID": 6} 2023-11-15T11:44:29.053 INFO:teuthology.provision.fog.smithi162:Scheduling deploy of ubuntu 22.04 2023-11-15T11:44:29.053 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /task/active None 2023-11-15T11:44:29.059 INFO:teuthology.provision.fog.smithi187:Scheduling deploy of ubuntu 22.04 2023-11-15T11:44:29.059 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None 2023-11-15T11:44:29.126 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /tasktype {"name": "deploy"} 2023-11-15T11:44:29.131 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /tasktype {"name": "deploy"} 2023-11-15T11:44:29.177 DEBUG:teuthology.provision.fog.smithi162:FOG request: POST /host/267/task {"taskTypeID": 1} 2023-11-15T11:44:29.181 DEBUG:teuthology.provision.fog.smithi187:FOG request: POST /host/292/task {"taskTypeID": 1} 2023-11-15T11:44:29.237 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None 2023-11-15T11:44:29.241 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /task/active None 2023-11-15T11:44:29.349 INFO:teuthology.orchestra.console.smithi187:Power off 2023-11-15T11:44:29.350 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-15T11:44:29.363 INFO:teuthology.orchestra.console.smithi162:Power off 2023-11-15T11:44:29.364 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-15T11:44:29.371 DEBUG:teuthology.orchestra.console.smithi187:power off output: Chassis Power Control: Down/Off 2023-11-15T11:44:29.372 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T11:44:29.388 DEBUG:teuthology.orchestra.console.smithi162:power off output: Chassis Power Control: Down/Off 2023-11-15T11:44:29.389 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T11:44:29.397 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2023-11-15T11:44:29.411 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2023-11-15T11:44:33.402 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T11:44:33.412 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T11:44:33.512 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2023-11-15T11:44:33.521 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2023-11-15T11:44:37.514 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T11:44:37.522 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T11:44:37.624 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2023-11-15T11:44:37.631 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2023-11-15T11:44:41.626 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T11:44:41.633 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T11:44:41.736 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is off 2023-11-15T11:44:41.742 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is off 2023-11-15T11:44:41.838 INFO:teuthology.orchestra.console.smithi187:Power off completed 2023-11-15T11:44:41.844 INFO:teuthology.orchestra.console.smithi162:Power off completed 2023-11-15T11:44:41.939 INFO:teuthology.orchestra.console.smithi187:Power on 2023-11-15T11:44:41.940 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-15T11:44:41.947 INFO:teuthology.orchestra.console.smithi162:Power on 2023-11-15T11:44:41.948 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-15T11:44:41.962 DEBUG:teuthology.orchestra.console.smithi187:power on output: Chassis Power Control: Up/On 2023-11-15T11:44:41.963 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T11:44:41.973 DEBUG:teuthology.orchestra.console.smithi162:power on output: Chassis Power Control: Up/On 2023-11-15T11:44:41.973 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T11:44:41.984 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is off 2023-11-15T11:44:41.995 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is off 2023-11-15T11:44:45.989 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T11:44:45.996 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T11:44:46.099 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2023-11-15T11:44:46.106 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2023-11-15T11:44:46.200 INFO:teuthology.orchestra.console.smithi187:Power on completed 2023-11-15T11:44:46.207 INFO:teuthology.orchestra.console.smithi162:Power on completed 2023-11-15T11:44:46.302 INFO:teuthology.provision.fog.smithi187:Waiting for deploy to finish 2023-11-15T11:44:46.302 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None 2023-11-15T11:44:46.308 INFO:teuthology.provision.fog.smithi162:Waiting for deploy to finish 2023-11-15T11:44:46.308 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /task/active None 2023-11-15T11:45:01.415 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None 2023-11-15T11:45:01.416 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /task/active None 2023-11-15T11:45:16.506 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /task/active None 2023-11-15T11:45:16.508 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None 2023-11-15T11:45:31.590 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None 2023-11-15T11:45:31.592 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /task/active None 2023-11-15T11:45:46.678 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /task/active None 2023-11-15T11:45:46.680 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None 2023-11-15T11:46:01.758 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None 2023-11-15T11:46:01.759 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /task/active None 2023-11-15T11:46:16.834 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /task/active None 2023-11-15T11:46:16.836 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None 2023-11-15T11:46:31.910 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None 2023-11-15T11:46:31.912 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /task/active None 2023-11-15T11:46:46.988 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None 2023-11-15T11:46:46.990 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /task/active None 2023-11-15T11:47:02.078 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /task/active None 2023-11-15T11:47:02.079 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None 2023-11-15T11:47:17.157 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None 2023-11-15T11:47:17.158 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /task/active None 2023-11-15T11:47:32.238 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /task/active None 2023-11-15T11:47:32.240 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None 2023-11-15T11:47:47.318 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /task/active None 2023-11-15T11:47:47.319 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None 2023-11-15T11:48:02.395 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /task/active None 2023-11-15T11:48:02.396 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None 2023-11-15T11:48:02.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:48:05.534 WARNING:teuthology.provision.fog.smithi187:[Errno None] Unable to connect to port 22 on 172.21.15.187 2023-11-15T11:48:11.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:48:14.618 WARNING:teuthology.provision.fog.smithi187:[Errno None] Unable to connect to port 22 on 172.21.15.187 2023-11-15T11:48:17.466 DEBUG:teuthology.provision.fog.smithi162:FOG request: GET /task/active None 2023-11-15T11:48:17.521 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:48:20.602 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-11-15T11:48:20.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:48:23.678 WARNING:teuthology.provision.fog.smithi187:[Errno None] Unable to connect to port 22 on 172.21.15.187 2023-11-15T11:48:26.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:48:29.658 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-11-15T11:48:29.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:48:32.730 WARNING:teuthology.provision.fog.smithi187:[Errno None] Unable to connect to port 22 on 172.21.15.187 2023-11-15T11:48:35.662 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:48:38.722 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-11-15T11:48:38.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:48:41.786 WARNING:teuthology.provision.fog.smithi187:[Errno None] Unable to connect to port 22 on 172.21.15.187 2023-11-15T11:48:44.727 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:48:47.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:48:47.802 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-11-15T11:48:50.842 WARNING:teuthology.provision.fog.smithi187:[Errno None] Unable to connect to port 22 on 172.21.15.187 2023-11-15T11:48:53.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:48:56.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:48:56.858 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-11-15T11:49:02.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:49:05.946 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-11-15T11:49:11.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:49:56.892 WARNING:teuthology.provision.fog.smithi187:timed out 2023-11-15T11:50:02.899 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:50:03.125 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-15T11:50:03.126 DEBUG:teuthology.orchestra.run.smithi187:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-15T11:50:03.580 INFO:teuthology.provision.fog.smithi187:Node is ready 2023-11-15T11:50:03.581 DEBUG:teuthology.orchestra.run.smithi187:> hostname 2023-11-15T11:50:03.627 INFO:teuthology.orchestra.run.smithi187.stdout:smithi187.front.sepia.ceph.com 2023-11-15T11:50:03.627 DEBUG:teuthology.orchestra.run.smithi187:> grep smithi187.front.sepia.ceph.com /etc/hosts 2023-11-15T11:50:03.673 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.15.187 smithi187.front.sepia.ceph.com smithi187 2023-11-15T11:50:03.674 DEBUG:teuthology.orchestra.run.smithi187:> sudo hostname smithi187 2023-11-15T11:50:03.803 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i -e 's/smithi187.front.sepia.ceph.com/smithi187/g' /etc/hosts 2023-11-15T11:50:03.818 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i -e 's/172.21.15.187/172.21.15.187/g' /etc/hosts 2023-11-15T11:50:03.873 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i -e 's/smithi187.front.sepia.ceph.com/smithi187/g' /etc/hostname 2023-11-15T11:50:03.929 DEBUG:teuthology.orchestra.run.smithi187:> sudo hostname smithi187 2023-11-15T11:50:03.989 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2023-11-15T11:50:04.038 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-15T11:50:04.038 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="Ubuntu" 2023-11-15T11:50:04.038 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="22.04" 2023-11-15T11:50:04.039 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-15T11:50:04.039 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_CODENAME=jammy 2023-11-15T11:50:04.039 INFO:teuthology.orchestra.run.smithi187.stdout:ID=ubuntu 2023-11-15T11:50:04.039 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE=debian 2023-11-15T11:50:04.039 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-15T11:50:04.040 INFO:teuthology.orchestra.run.smithi187.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-15T11:50:04.040 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-15T11:50:04.040 INFO:teuthology.orchestra.run.smithi187.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-15T11:50:04.041 INFO:teuthology.orchestra.run.smithi187.stdout:UBUNTU_CODENAME=jammy 2023-11-15T11:50:04.041 INFO:teuthology.provision.fog.smithi187:Deploy complete! 2023-11-15T11:50:04.043 DEBUG:teuthology.parallel:result is None 2023-11-15T11:50:11.962 WARNING:teuthology.provision.fog.smithi162:timed out 2023-11-15T11:50:17.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:50:18.298 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-15T11:50:18.298 DEBUG:teuthology.orchestra.run.smithi162:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-15T11:50:18.695 INFO:teuthology.provision.fog.smithi162:Node is ready 2023-11-15T11:50:18.696 DEBUG:teuthology.orchestra.run.smithi162:> hostname 2023-11-15T11:50:18.741 INFO:teuthology.orchestra.run.smithi162.stdout:smithi162.front.sepia.ceph.com 2023-11-15T11:50:18.741 DEBUG:teuthology.orchestra.run.smithi162:> grep smithi162.front.sepia.ceph.com /etc/hosts 2023-11-15T11:50:18.789 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162 smithi162.front.sepia.ceph.com smithi162 2023-11-15T11:50:18.789 DEBUG:teuthology.orchestra.run.smithi162:> sudo hostname smithi162 2023-11-15T11:50:18.914 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i -e 's/smithi162.front.sepia.ceph.com/smithi162/g' /etc/hosts 2023-11-15T11:50:18.929 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i -e 's/172.21.15.162/172.21.15.162/g' /etc/hosts 2023-11-15T11:50:18.986 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i -e 's/smithi162.front.sepia.ceph.com/smithi162/g' /etc/hostname 2023-11-15T11:50:19.041 DEBUG:teuthology.orchestra.run.smithi162:> sudo hostname smithi162 2023-11-15T11:50:19.096 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2023-11-15T11:50:19.141 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-15T11:50:19.141 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="Ubuntu" 2023-11-15T11:50:19.141 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="22.04" 2023-11-15T11:50:19.142 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-15T11:50:19.142 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_CODENAME=jammy 2023-11-15T11:50:19.142 INFO:teuthology.orchestra.run.smithi162.stdout:ID=ubuntu 2023-11-15T11:50:19.142 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE=debian 2023-11-15T11:50:19.142 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-15T11:50:19.143 INFO:teuthology.orchestra.run.smithi162.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-15T11:50:19.143 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-15T11:50:19.143 INFO:teuthology.orchestra.run.smithi162.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-15T11:50:19.143 INFO:teuthology.orchestra.run.smithi162.stdout:UBUNTU_CODENAME=jammy 2023-11-15T11:50:19.143 INFO:teuthology.provision.fog.smithi162:Deploy complete! 2023-11-15T11:50:19.144 DEBUG:teuthology.parallel:result is None 2023-11-15T11:50:19.355 INFO:teuthology.lock.ops:Checking smithi162.front.sepia.ceph.com 2023-11-15T11:50:19.356 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-15T11:50:19.379 INFO:teuthology.lock.ops:Checking smithi187.front.sepia.ceph.com 2023-11-15T11:50:19.380 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-15T11:50:19.405 INFO:teuthology.lock.ops:Updating [smithi162.front.sepia.ceph.com]: set os type and version on server 2023-11-15T11:50:19.406 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-11-15T11:50:19.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:50:19.483 DEBUG:teuthology.orchestra.run.smithi162:> true 2023-11-15T11:50:19.598 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-11-15T11:50:19.599 DEBUG:teuthology.orchestra.run.smithi162:> uname -m 2023-11-15T11:50:19.644 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2023-11-15T11:50:19.644 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2023-11-15T11:50:19.697 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-15T11:50:19.697 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="Ubuntu" 2023-11-15T11:50:19.698 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="22.04" 2023-11-15T11:50:19.698 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-15T11:50:19.698 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_CODENAME=jammy 2023-11-15T11:50:19.699 INFO:teuthology.orchestra.run.smithi162.stdout:ID=ubuntu 2023-11-15T11:50:19.699 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE=debian 2023-11-15T11:50:19.700 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-15T11:50:19.700 INFO:teuthology.orchestra.run.smithi162.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-15T11:50:19.701 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-15T11:50:19.701 INFO:teuthology.orchestra.run.smithi162.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-15T11:50:19.701 INFO:teuthology.orchestra.run.smithi162.stdout:UBUNTU_CODENAME=jammy 2023-11-15T11:50:19.701 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2023-11-15T11:50:19.722 INFO:teuthology.lock.ops:Updating [smithi187.front.sepia.ceph.com]: set os type and version on server 2023-11-15T11:50:19.722 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2023-11-15T11:50:19.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T11:50:19.801 DEBUG:teuthology.orchestra.run.smithi187:> true 2023-11-15T11:50:20.038 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2023-11-15T11:50:20.038 DEBUG:teuthology.orchestra.run.smithi187:> uname -m 2023-11-15T11:50:20.090 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2023-11-15T11:50:20.091 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2023-11-15T11:50:20.138 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-15T11:50:20.138 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="Ubuntu" 2023-11-15T11:50:20.139 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="22.04" 2023-11-15T11:50:20.139 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-15T11:50:20.139 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_CODENAME=jammy 2023-11-15T11:50:20.140 INFO:teuthology.orchestra.run.smithi187.stdout:ID=ubuntu 2023-11-15T11:50:20.140 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE=debian 2023-11-15T11:50:20.141 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-15T11:50:20.141 INFO:teuthology.orchestra.run.smithi187.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-15T11:50:20.141 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-15T11:50:20.141 INFO:teuthology.orchestra.run.smithi187.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-15T11:50:20.141 INFO:teuthology.orchestra.run.smithi187.stdout:UBUNTU_CODENAME=jammy 2023-11-15T11:50:20.142 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2023-11-15T11:50:20.163 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T11:50:20.254 INFO:teuthology.dispatcher.supervisor:Running job 7459116 2023-11-15T11:50:20.255 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_rfriedma@teuthology --archive /home/teuthworker/archive/rfriedma-2023-11-15_11:32:03-rados:thrash-wip-rf-force-scrub-distro-default-smithi/7459116 --name rfriedma-2023-11-15_11:32:03-rados:thrash-wip-rf-force-scrub-distro-default-smithi --description rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/cache-snaps} -- /home/teuthworker/archive/rfriedma-2023-11-15_11:32:03-rados:thrash-wip-rf-force-scrub-distro-default-smithi/7459116/orig.config.yaml 2023-11-15T11:50:20.258 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rfriedma-2023-11-15_11:32:03-rados:thrash-wip-rf-force-scrub-distro-default-smithi/7459116 2023-11-15T11:50:20.259 INFO:teuthology.dispatcher.supervisor:Job PID: 1948520 2023-11-15T11:50:20.259 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-15T11:52:20.360 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T11:54:20.523 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T11:56:20.691 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T11:58:20.863 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T12:00:21.039 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T12:02:21.216 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T12:02:21.280 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-11-15T12:02:21.402 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-15T12:02:21.402 INFO:teuthology.nuke:Checking targets against current locks 2023-11-15T12:02:21.451 DEBUG:teuthology.nuke:shortname: smithi162 2023-11-15T12:02:21.451 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-15T12:02:21.474 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rfriedma-2023-11-15_11:32:03-rados:thrash-wip-rf-force-scrub-distro-default-smithi/7459116', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-15 11:44:27.873956', 'locked_by': 'scheduled_rfriedma@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJWzmaPNDzIsmuv3X+n3DkIFS7lRUw0dXMdlsojCnlyWJQjRDoZOQqwe/RyDVsKWdsPBpSCIX9bV8PbEcVJWi40='} 2023-11-15T12:02:21.478 DEBUG:teuthology.nuke:shortname: smithi187 2023-11-15T12:02:21.479 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-15T12:02:21.498 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rfriedma-2023-11-15_11:32:03-rados:thrash-wip-rf-force-scrub-distro-default-smithi/7459116', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-15 11:44:27.875509', 'locked_by': 'scheduled_rfriedma@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE8AYyNfn7Texr2rEhcbQY/M3gi6IjvGkM6Mrio+hj3zT7QJVDaLA2xqppEx2ARGKOQp0gPI8CMBnhemUexs3Vw='} 2023-11-15T12:02:21.526 INFO:teuthology.orchestra.console.smithi162:Power off 2023-11-15T12:02:21.526 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-15T12:02:21.549 DEBUG:teuthology.orchestra.console.smithi162:power off output: Chassis Power Control: Down/Off 2023-11-15T12:02:21.550 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T12:02:21.566 INFO:teuthology.orchestra.console.smithi187:Power off 2023-11-15T12:02:21.566 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-15T12:02:21.575 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is off 2023-11-15T12:02:21.589 DEBUG:teuthology.orchestra.console.smithi187:power off output: Chassis Power Control: Down/Off 2023-11-15T12:02:21.590 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T12:02:21.614 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is off 2023-11-15T12:02:21.676 INFO:teuthology.orchestra.console.smithi162:Power off completed 2023-11-15T12:02:21.716 INFO:teuthology.orchestra.console.smithi187:Power off completed 2023-11-15T12:02:21.829 INFO:teuthology.lock.ops:unlocked: smithi162.front.sepia.ceph.com 2023-11-15T12:02:21.862 INFO:teuthology.lock.ops:unlocked: smithi187.front.sepia.ceph.com