2023-09-22T19:26:09.547 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-22T19:26:09.548 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T19:26:09.679 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f37b616d670>, signals=[15]) 2023-09-22T19:26:09.679 INFO:teuthology.lock.ops:Start node 'smithi112.front.sepia.ceph.com' reimaging 2023-09-22T19:26:09.679 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: reset os type and version on server 2023-09-22T19:26:09.680 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2023-09-22T19:26:09.701 INFO:teuthology.lock.ops:Start node 'smithi119.front.sepia.ceph.com' reimaging 2023-09-22T19:26:09.701 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: reset os type and version on server 2023-09-22T19:26:09.701 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2023-09-22T19:26:09.702 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /host {"name": "smithi112"} 2023-09-22T19:26:09.725 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /host {"name": "smithi119"} 2023-09-22T19:26:09.796 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /image {"name": "smithi_ubuntu_22.04"} 2023-09-22T19:26:09.802 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /image {"name": "smithi_ubuntu_22.04"} 2023-09-22T19:26:09.842 DEBUG:teuthology.provision.fog.smithi112:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-09-22T19:26:09.843 DEBUG:teuthology.provision.fog.smithi112:FOG request: PUT /host/217 {"imageID": 6} 2023-09-22T19:26:09.844 DEBUG:teuthology.provision.fog.smithi119:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-09-22T19:26:09.844 DEBUG:teuthology.provision.fog.smithi119:FOG request: PUT /host/224 {"imageID": 6} 2023-09-22T19:26:09.889 INFO:teuthology.provision.fog.smithi119:Scheduling deploy of ubuntu 22.04 2023-09-22T19:26:09.889 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-09-22T19:26:09.895 INFO:teuthology.provision.fog.smithi112:Scheduling deploy of ubuntu 22.04 2023-09-22T19:26:09.895 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-09-22T19:26:09.937 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /tasktype {"name": "deploy"} 2023-09-22T19:26:09.940 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /tasktype {"name": "deploy"} 2023-09-22T19:26:09.970 DEBUG:teuthology.provision.fog.smithi112:FOG request: POST /host/217/task {"taskTypeID": 1} 2023-09-22T19:26:09.979 DEBUG:teuthology.provision.fog.smithi119:FOG request: POST /host/224/task {"taskTypeID": 1} 2023-09-22T19:26:10.035 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-09-22T19:26:10.048 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-09-22T19:26:10.117 INFO:teuthology.orchestra.console.smithi119:Power off 2023-09-22T19:26:10.117 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-22T19:26:10.130 INFO:teuthology.orchestra.console.smithi112:Power off 2023-09-22T19:26:10.130 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-22T19:26:10.140 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off 2023-09-22T19:26:10.141 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-22T19:26:10.150 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off 2023-09-22T19:26:10.150 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-22T19:26:10.161 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2023-09-22T19:26:10.171 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2023-09-22T19:26:10.271 INFO:teuthology.orchestra.console.smithi112:Power off completed 2023-09-22T19:26:10.372 INFO:teuthology.orchestra.console.smithi112:Power on 2023-09-22T19:26:10.372 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-22T19:26:10.396 DEBUG:teuthology.orchestra.console.smithi112:power on output: Chassis Power Control: Up/On 2023-09-22T19:26:10.396 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-22T19:26:10.415 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2023-09-22T19:26:14.162 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-22T19:26:14.270 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2023-09-22T19:26:14.417 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-22T19:26:14.525 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2023-09-22T19:26:18.274 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-22T19:26:18.382 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2023-09-22T19:26:18.526 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-22T19:26:18.634 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2023-09-22T19:26:18.735 INFO:teuthology.orchestra.console.smithi112:Power on completed 2023-09-22T19:26:18.835 INFO:teuthology.provision.fog.smithi112:Waiting for deploy to finish 2023-09-22T19:26:18.836 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-09-22T19:26:22.386 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-22T19:26:22.494 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off 2023-09-22T19:26:22.595 INFO:teuthology.orchestra.console.smithi119:Power off completed 2023-09-22T19:26:22.696 INFO:teuthology.orchestra.console.smithi119:Power on 2023-09-22T19:26:22.696 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-22T19:26:22.717 DEBUG:teuthology.orchestra.console.smithi119:power on output: Chassis Power Control: Up/On 2023-09-22T19:26:22.717 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-22T19:26:22.736 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off 2023-09-22T19:26:26.741 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-22T19:26:26.849 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off 2023-09-22T19:26:30.854 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-22T19:26:30.962 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2023-09-22T19:26:31.063 INFO:teuthology.orchestra.console.smithi119:Power on completed 2023-09-22T19:26:31.163 INFO:teuthology.provision.fog.smithi119:Waiting for deploy to finish 2023-09-22T19:26:31.164 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-09-22T19:26:33.893 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-09-22T19:26:46.234 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-09-22T19:26:48.962 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-09-22T19:27:01.328 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-09-22T19:27:04.036 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-09-22T19:27:16.403 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-09-22T19:27:19.108 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-09-22T19:27:31.486 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-09-22T19:27:34.181 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-09-22T19:27:46.554 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-09-22T19:27:49.255 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-09-22T19:28:01.626 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-09-22T19:28:04.326 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-09-22T19:28:16.714 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-09-22T19:28:19.402 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-09-22T19:28:31.806 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-09-22T19:28:34.475 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-09-22T19:28:46.893 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-09-22T19:28:49.558 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-09-22T19:29:01.969 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-09-22T19:29:04.632 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-09-22T19:29:17.034 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-09-22T19:29:19.702 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-09-22T19:29:32.114 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-09-22T19:29:34.763 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-09-22T19:29:34.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:29:47.178 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-09-22T19:29:47.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:29:53.274 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-09-22T19:29:59.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:30:02.334 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-09-22T19:30:05.562 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-09-22T19:30:08.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:30:11.418 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-09-22T19:30:11.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:30:14.618 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-09-22T19:30:17.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:30:20.474 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-09-22T19:30:20.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:30:23.706 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-09-22T19:30:26.478 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:30:29.530 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-09-22T19:30:29.708 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:30:32.766 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-09-22T19:30:35.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:30:38.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:31:07.007 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-09-22T19:31:13.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:31:13.013 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-09-22T19:31:19.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:31:19.020 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-09-22T19:31:25.029 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:31:25.037 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-09-22T19:31:31.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:31:31.040 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-09-22T19:31:37.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:31:37.259 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-22T19:31:37.259 DEBUG:teuthology.orchestra.run.smithi112:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-22T19:31:37.800 INFO:teuthology.provision.fog.smithi112:Node is ready 2023-09-22T19:31:37.801 DEBUG:teuthology.orchestra.run.smithi112:> hostname 2023-09-22T19:31:37.846 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112.front.sepia.ceph.com 2023-09-22T19:31:37.846 DEBUG:teuthology.orchestra.run.smithi112:> grep smithi112.front.sepia.ceph.com /etc/hosts 2023-09-22T19:31:37.894 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112 smithi112.front.sepia.ceph.com smithi112 2023-09-22T19:31:37.894 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname smithi112 2023-09-22T19:31:38.023 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/smithi112.front.sepia.ceph.com/smithi112/g' /etc/hosts 2023-09-22T19:31:38.037 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/172.21.15.112/172.21.15.112/g' /etc/hosts 2023-09-22T19:31:38.089 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/smithi112.front.sepia.ceph.com/smithi112/g' /etc/hostname 2023-09-22T19:31:38.147 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname smithi112 2023-09-22T19:31:38.201 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2023-09-22T19:31:38.250 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-09-22T19:31:38.250 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="Ubuntu" 2023-09-22T19:31:38.250 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="22.04" 2023-09-22T19:31:38.250 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-09-22T19:31:38.250 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_CODENAME=jammy 2023-09-22T19:31:38.250 INFO:teuthology.orchestra.run.smithi112.stdout:ID=ubuntu 2023-09-22T19:31:38.250 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE=debian 2023-09-22T19:31:38.250 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://www.ubuntu.com/" 2023-09-22T19:31:38.250 INFO:teuthology.orchestra.run.smithi112.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-09-22T19:31:38.251 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-09-22T19:31:38.251 INFO:teuthology.orchestra.run.smithi112.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-09-22T19:31:38.251 INFO:teuthology.orchestra.run.smithi112.stdout:UBUNTU_CODENAME=jammy 2023-09-22T19:31:38.251 INFO:teuthology.provision.fog.smithi112:Deploy complete! 2023-09-22T19:31:38.252 DEBUG:teuthology.parallel:result is None 2023-09-22T19:31:38.773 WARNING:teuthology.provision.fog.smithi119:timed out 2023-09-22T19:31:44.780 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:31:44.983 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-22T19:31:44.983 DEBUG:teuthology.orchestra.run.smithi119:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-22T19:31:45.472 INFO:teuthology.provision.fog.smithi119:Node is ready 2023-09-22T19:31:45.677 DEBUG:teuthology.orchestra.run.smithi119:> hostname 2023-09-22T19:31:45.683 INFO:teuthology.orchestra.run.smithi119.stdout:smithi119.front.sepia.ceph.com 2023-09-22T19:31:45.683 DEBUG:teuthology.orchestra.run.smithi119:> grep smithi119.front.sepia.ceph.com /etc/hosts 2023-09-22T19:31:45.730 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119 smithi119.front.sepia.ceph.com smithi119 2023-09-22T19:31:45.743 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname smithi119 2023-09-22T19:31:45.898 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i -e 's/smithi119.front.sepia.ceph.com/smithi119/g' /etc/hosts 2023-09-22T19:31:46.295 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i -e 's/172.21.15.119/172.21.15.119/g' /etc/hosts 2023-09-22T19:31:46.350 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i -e 's/smithi119.front.sepia.ceph.com/smithi119/g' /etc/hostname 2023-09-22T19:31:46.410 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname smithi119 2023-09-22T19:31:46.468 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2023-09-22T19:31:46.517 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-09-22T19:31:46.518 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Ubuntu" 2023-09-22T19:31:46.518 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="22.04" 2023-09-22T19:31:46.518 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-09-22T19:31:46.518 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_CODENAME=jammy 2023-09-22T19:31:46.518 INFO:teuthology.orchestra.run.smithi119.stdout:ID=ubuntu 2023-09-22T19:31:46.518 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE=debian 2023-09-22T19:31:46.518 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.ubuntu.com/" 2023-09-22T19:31:46.518 INFO:teuthology.orchestra.run.smithi119.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-09-22T19:31:46.518 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-09-22T19:31:46.518 INFO:teuthology.orchestra.run.smithi119.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-09-22T19:31:46.518 INFO:teuthology.orchestra.run.smithi119.stdout:UBUNTU_CODENAME=jammy 2023-09-22T19:31:46.519 INFO:teuthology.provision.fog.smithi119:Deploy complete! 2023-09-22T19:31:46.519 DEBUG:teuthology.parallel:result is None 2023-09-22T19:31:46.700 INFO:teuthology.lock.ops:Checking smithi112.front.sepia.ceph.com 2023-09-22T19:31:46.700 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-22T19:31:46.722 INFO:teuthology.lock.ops:Checking smithi119.front.sepia.ceph.com 2023-09-22T19:31:46.723 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-22T19:31:46.740 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: set os type and version on server 2023-09-22T19:31:46.740 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-09-22T19:31:46.741 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:31:46.814 DEBUG:teuthology.orchestra.run.smithi112:> true 2023-09-22T19:31:46.922 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-09-22T19:31:46.922 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2023-09-22T19:31:46.973 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2023-09-22T19:31:46.973 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2023-09-22T19:31:47.021 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-09-22T19:31:47.021 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="Ubuntu" 2023-09-22T19:31:47.021 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="22.04" 2023-09-22T19:31:47.021 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-09-22T19:31:47.021 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_CODENAME=jammy 2023-09-22T19:31:47.021 INFO:teuthology.orchestra.run.smithi112.stdout:ID=ubuntu 2023-09-22T19:31:47.021 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE=debian 2023-09-22T19:31:47.022 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://www.ubuntu.com/" 2023-09-22T19:31:47.022 INFO:teuthology.orchestra.run.smithi112.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-09-22T19:31:47.022 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-09-22T19:31:47.022 INFO:teuthology.orchestra.run.smithi112.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-09-22T19:31:47.022 INFO:teuthology.orchestra.run.smithi112.stdout:UBUNTU_CODENAME=jammy 2023-09-22T19:31:47.022 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2023-09-22T19:31:47.048 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: set os type and version on server 2023-09-22T19:31:47.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-09-22T19:31:47.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-22T19:31:47.124 DEBUG:teuthology.orchestra.run.smithi119:> true 2023-09-22T19:31:47.234 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-09-22T19:31:47.235 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2023-09-22T19:31:47.285 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2023-09-22T19:31:47.285 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2023-09-22T19:31:47.334 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-09-22T19:31:47.334 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Ubuntu" 2023-09-22T19:31:47.334 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="22.04" 2023-09-22T19:31:47.334 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-09-22T19:31:47.334 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_CODENAME=jammy 2023-09-22T19:31:47.334 INFO:teuthology.orchestra.run.smithi119.stdout:ID=ubuntu 2023-09-22T19:31:47.335 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE=debian 2023-09-22T19:31:47.335 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.ubuntu.com/" 2023-09-22T19:31:47.335 INFO:teuthology.orchestra.run.smithi119.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-09-22T19:31:47.335 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-09-22T19:31:47.335 INFO:teuthology.orchestra.run.smithi119.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-09-22T19:31:47.335 INFO:teuthology.orchestra.run.smithi119.stdout:UBUNTU_CODENAME=jammy 2023-09-22T19:31:47.336 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2023-09-22T19:31:47.357 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T19:31:47.451 INFO:teuthology.dispatcher.supervisor:Running job 7401228 2023-09-22T19:31:47.451 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology -v --owner scheduled_cbodley@teuthology --archive /home/teuthworker/archive/cbodley-2023-09-22_18:26:40-rgw-main-distro-default-smithi/7401228 --name cbodley-2023-09-22_18:26:40-rgw-main-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{ubuntu_latest} tasks/test_multi} -- /home/teuthworker/archive/cbodley-2023-09-22_18:26:40-rgw-main-distro-default-smithi/7401228/orig.config.yaml 2023-09-22T19:31:47.454 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/cbodley-2023-09-22_18:26:40-rgw-main-distro-default-smithi/7401228 2023-09-22T19:31:47.454 INFO:teuthology.dispatcher.supervisor:Job PID: 2089832 2023-09-22T19:31:47.454 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-09-22T19:33:47.555 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T19:35:47.723 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T19:37:47.793 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T19:39:47.940 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T19:41:48.097 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T19:43:48.256 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T19:45:48.390 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T19:47:48.556 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T19:49:48.711 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T19:51:48.871 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T19:53:49.043 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T19:55:49.207 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T19:57:49.379 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T19:59:49.473 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:01:49.630 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:03:49.791 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:05:49.955 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:07:50.129 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:09:50.294 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:11:50.455 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:13:50.625 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:15:50.740 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:17:50.884 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:19:51.046 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:21:51.210 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:23:51.375 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:25:51.543 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:27:51.703 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:29:51.862 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:31:52.027 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:33:52.189 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:35:52.345 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:37:52.512 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:39:52.670 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:41:52.831 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:43:53.059 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:45:53.300 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:47:53.461 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:49:53.595 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:51:53.759 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:53:53.928 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:55:54.074 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:57:54.251 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T20:59:54.401 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:01:54.564 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:03:54.639 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:05:54.812 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:07:54.869 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:09:55.042 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:11:55.213 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:13:55.375 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:15:55.546 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:17:55.708 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:19:55.866 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:21:56.063 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:23:56.215 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:25:56.384 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:27:56.535 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:29:56.730 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:31:56.863 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:33:56.943 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:35:57.111 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:37:57.283 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:39:57.446 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:41:57.603 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:43:57.767 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:45:57.851 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:47:58.018 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:49:58.179 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:51:58.335 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:53:58.471 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-22T21:53:58.539 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-09-22T21:53:58.712 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-09-22T21:53:58.712 INFO:teuthology.nuke:Checking targets against current locks 2023-09-22T21:53:58.758 DEBUG:teuthology.nuke:shortname: smithi112 2023-09-22T21:53:58.758 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-22T21:53:58.780 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2023-09-22_18:26:40-rgw-main-distro-default-smithi/7401228', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-22 19:26:08.673214', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBC4DPlJAL5d4THoh7xhk01jRiDnM4AB0ZcXcMIrPcWapzcJ8OSngR/AWE1mlODIKZ+m5aSDDe5xpKKr/3DOQo0='} 2023-09-22T21:53:58.785 DEBUG:teuthology.nuke:shortname: smithi119 2023-09-22T21:53:58.785 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-22T21:53:58.805 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2023-09-22_18:26:40-rgw-main-distro-default-smithi/7401228', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-22 19:26:08.671587', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBObXPQ1xMIAe8sPfLX0KNsrzQ4UENyYPWFKnTgV0/kjM6qbq8Db+vZFfV9fgYnZawSwcJMw/Vw7Rt808c+mVs4g='} 2023-09-22T21:53:58.826 INFO:teuthology.orchestra.console.smithi112:Power off 2023-09-22T21:53:58.826 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-22T21:53:58.848 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off 2023-09-22T21:53:58.848 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-22T21:53:58.861 INFO:teuthology.orchestra.console.smithi119:Power off 2023-09-22T21:53:58.862 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-22T21:53:58.868 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2023-09-22T21:53:58.881 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off 2023-09-22T21:53:58.881 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-22T21:53:58.901 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off 2023-09-22T21:53:58.969 INFO:teuthology.orchestra.console.smithi112:Power off completed 2023-09-22T21:53:59.002 INFO:teuthology.orchestra.console.smithi119:Power off completed 2023-09-22T21:53:59.121 INFO:teuthology.lock.ops:unlocked: smithi112.front.sepia.ceph.com 2023-09-22T21:53:59.280 INFO:teuthology.lock.ops:unlocked: smithi119.front.sepia.ceph.com