2023-08-21T18:03:00.721 INFO:root:teuthology version: 0.0.1.dev185+g449a1bc 2023-08-21T18:03:00.722 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:03:00.847 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f92c4e88f70>, signals=[15]) 2023-08-21T18:03:00.848 INFO:teuthology.lock.ops:Start node 'smithi044.front.sepia.ceph.com' reimaging 2023-08-21T18:03:00.849 INFO:teuthology.lock.ops:Updating [smithi044.front.sepia.ceph.com]: reset os type and version on server 2023-08-21T18:03:00.849 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2023-08-21T18:03:00.868 INFO:teuthology.lock.ops:Start node 'smithi169.front.sepia.ceph.com' reimaging 2023-08-21T18:03:00.868 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: reset os type and version on server 2023-08-21T18:03:00.869 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2023-08-21T18:03:00.870 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /host {"name": "smithi044"} 2023-08-21T18:03:00.901 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /host {"name": "smithi169"} 2023-08-21T18:03:00.937 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /image {"name": "smithi_ubuntu_22.04"} 2023-08-21T18:03:00.966 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /image {"name": "smithi_ubuntu_22.04"} 2023-08-21T18:03:00.980 DEBUG:teuthology.provision.fog.smithi044:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-08-21T18:03:00.980 DEBUG:teuthology.provision.fog.smithi044:FOG request: PUT /host/149 {"imageID": 6} 2023-08-21T18:03:01.011 DEBUG:teuthology.provision.fog.smithi169:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-08-21T18:03:01.012 DEBUG:teuthology.provision.fog.smithi169:FOG request: PUT /host/274 {"imageID": 6} 2023-08-21T18:03:01.024 INFO:teuthology.provision.fog.smithi044:Scheduling deploy of ubuntu 22.04 2023-08-21T18:03:01.024 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-21T18:03:01.059 INFO:teuthology.provision.fog.smithi169:Scheduling deploy of ubuntu 22.04 2023-08-21T18:03:01.059 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:03:01.062 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /tasktype {"name": "deploy"} 2023-08-21T18:03:01.092 DEBUG:teuthology.provision.fog.smithi044:FOG request: POST /host/149/task {"taskTypeID": 1} 2023-08-21T18:03:01.099 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /tasktype {"name": "deploy"} 2023-08-21T18:03:01.142 DEBUG:teuthology.provision.fog.smithi169:FOG request: POST /host/274/task {"taskTypeID": 1} 2023-08-21T18:03:01.157 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-21T18:03:01.200 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:03:01.231 INFO:teuthology.orchestra.console.smithi044:Power off 2023-08-21T18:03:01.232 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-21T18:03:01.255 DEBUG:teuthology.orchestra.console.smithi044:power off output: Chassis Power Control: Down/Off 2023-08-21T18:03:01.255 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-21T18:03:01.276 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2023-08-21T18:03:01.288 INFO:teuthology.orchestra.console.smithi169:Power off 2023-08-21T18:03:01.289 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-21T18:03:01.313 DEBUG:teuthology.orchestra.console.smithi169:power off output: Chassis Power Control: Down/Off 2023-08-21T18:03:01.314 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-21T18:03:01.335 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2023-08-21T18:03:01.378 INFO:teuthology.orchestra.console.smithi044:Power off completed 2023-08-21T18:03:01.478 INFO:teuthology.orchestra.console.smithi044:Power on 2023-08-21T18:03:01.479 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-21T18:03:01.498 DEBUG:teuthology.orchestra.console.smithi044:power on output: Chassis Power Control: Up/On 2023-08-21T18:03:01.498 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-21T18:03:01.520 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2023-08-21T18:03:05.340 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-21T18:03:05.450 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2023-08-21T18:03:05.522 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-21T18:03:05.630 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2023-08-21T18:03:09.452 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-21T18:03:09.561 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off 2023-08-21T18:03:09.631 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-21T18:03:09.662 INFO:teuthology.orchestra.console.smithi169:Power off completed 2023-08-21T18:03:09.740 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2023-08-21T18:03:09.763 INFO:teuthology.orchestra.console.smithi169:Power on 2023-08-21T18:03:09.764 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-21T18:03:09.783 DEBUG:teuthology.orchestra.console.smithi169:power on output: Chassis Power Control: Up/On 2023-08-21T18:03:09.784 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-21T18:03:09.803 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off 2023-08-21T18:03:09.840 INFO:teuthology.orchestra.console.smithi044:Power on completed 2023-08-21T18:03:09.941 INFO:teuthology.provision.fog.smithi044:Waiting for deploy to finish 2023-08-21T18:03:09.941 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-21T18:03:13.808 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-21T18:03:13.916 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off 2023-08-21T18:03:17.921 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-21T18:03:18.030 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2023-08-21T18:03:18.131 INFO:teuthology.orchestra.console.smithi169:Power on completed 2023-08-21T18:03:18.232 INFO:teuthology.provision.fog.smithi169:Waiting for deploy to finish 2023-08-21T18:03:18.232 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:03:25.003 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-21T18:03:33.288 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:03:40.061 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-21T18:03:48.354 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:03:55.121 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-21T18:04:03.413 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:04:10.191 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-21T18:04:18.475 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:04:25.245 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-21T18:04:33.535 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:04:40.300 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-21T18:04:48.591 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:04:55.365 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-21T18:05:03.652 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:05:10.415 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-21T18:05:18.704 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:05:25.466 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-21T18:05:33.756 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:05:40.516 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-21T18:05:48.803 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:05:55.573 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-21T18:06:03.861 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:06:10.615 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-21T18:06:18.935 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:06:25.668 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-21T18:06:33.990 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:06:40.724 DEBUG:teuthology.provision.fog.smithi044:FOG request: GET /task/active None 2023-08-21T18:06:40.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-21T18:06:49.040 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:06:59.187 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2023-08-21T18:07:04.093 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None 2023-08-21T18:07:04.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-21T18:07:05.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-21T18:07:08.240 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2023-08-21T18:07:14.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-21T18:07:17.331 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2023-08-21T18:07:22.480 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-08-21T18:07:23.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-21T18:07:26.384 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2023-08-21T18:07:28.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-21T18:07:31.536 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-08-21T18:07:32.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-21T18:07:37.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-21T18:07:40.592 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-08-21T18:07:46.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-21T18:07:49.680 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169 2023-08-21T18:07:55.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-21T18:08:32.421 WARNING:teuthology.provision.fog.smithi044:timed out 2023-08-21T18:08:38.429 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-21T18:08:38.602 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-21T18:08:38.603 DEBUG:teuthology.orchestra.run.smithi044:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-21T18:08:39.095 INFO:teuthology.provision.fog.smithi044:Node is ready 2023-08-21T18:08:39.096 DEBUG:teuthology.orchestra.run.smithi044:> hostname 2023-08-21T18:08:39.141 INFO:teuthology.orchestra.run.smithi044.stdout:smithi044.front.sepia.ceph.com 2023-08-21T18:08:39.141 DEBUG:teuthology.orchestra.run.smithi044:> grep smithi044.front.sepia.ceph.com /etc/hosts 2023-08-21T18:08:39.189 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44 smithi044.front.sepia.ceph.com smithi044 2023-08-21T18:08:39.189 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname smithi044 2023-08-21T18:08:39.335 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/smithi044.front.sepia.ceph.com/smithi044/g' /etc/hosts 2023-08-21T18:08:39.349 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/172.21.15.44/172.21.15.44/g' /etc/hosts 2023-08-21T18:08:39.409 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/smithi044.front.sepia.ceph.com/smithi044/g' /etc/hostname 2023-08-21T18:08:39.469 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname smithi044 2023-08-21T18:08:39.528 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2023-08-21T18:08:39.572 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-08-21T18:08:39.573 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="Ubuntu" 2023-08-21T18:08:39.573 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="22.04" 2023-08-21T18:08:39.573 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-08-21T18:08:39.573 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_CODENAME=jammy 2023-08-21T18:08:39.573 INFO:teuthology.orchestra.run.smithi044.stdout:ID=ubuntu 2023-08-21T18:08:39.573 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE=debian 2023-08-21T18:08:39.574 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://www.ubuntu.com/" 2023-08-21T18:08:39.574 INFO:teuthology.orchestra.run.smithi044.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-08-21T18:08:39.574 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-08-21T18:08:39.574 INFO:teuthology.orchestra.run.smithi044.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-08-21T18:08:39.574 INFO:teuthology.orchestra.run.smithi044.stdout:UBUNTU_CODENAME=jammy 2023-08-21T18:08:39.575 INFO:teuthology.provision.fog.smithi044:Deploy complete! 2023-08-21T18:08:39.576 DEBUG:teuthology.parallel:result is None 2023-08-21T18:08:55.704 WARNING:teuthology.provision.fog.smithi169:timed out 2023-08-21T18:09:01.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-21T18:09:01.988 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-21T18:09:01.988 DEBUG:teuthology.orchestra.run.smithi169:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-21T18:09:02.731 INFO:teuthology.provision.fog.smithi169:Node is ready 2023-08-21T18:09:02.732 DEBUG:teuthology.orchestra.run.smithi169:> hostname 2023-08-21T18:09:02.778 INFO:teuthology.orchestra.run.smithi169.stdout:smithi169.front.sepia.ceph.com 2023-08-21T18:09:02.779 DEBUG:teuthology.orchestra.run.smithi169:> grep smithi169.front.sepia.ceph.com /etc/hosts 2023-08-21T18:09:02.829 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.15.169 smithi169.front.sepia.ceph.com smithi169 2023-08-21T18:09:02.829 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname smithi169 2023-08-21T18:09:03.001 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/smithi169.front.sepia.ceph.com/smithi169/g' /etc/hosts 2023-08-21T18:09:03.015 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/172.21.15.169/172.21.15.169/g' /etc/hosts 2023-08-21T18:09:03.103 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/smithi169.front.sepia.ceph.com/smithi169/g' /etc/hostname 2023-08-21T18:09:03.130 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname smithi169 2023-08-21T18:09:03.183 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2023-08-21T18:09:03.232 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-08-21T18:09:03.232 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="Ubuntu" 2023-08-21T18:09:03.232 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="22.04" 2023-08-21T18:09:03.232 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-08-21T18:09:03.232 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_CODENAME=jammy 2023-08-21T18:09:03.233 INFO:teuthology.orchestra.run.smithi169.stdout:ID=ubuntu 2023-08-21T18:09:03.233 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE=debian 2023-08-21T18:09:03.233 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://www.ubuntu.com/" 2023-08-21T18:09:03.233 INFO:teuthology.orchestra.run.smithi169.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-08-21T18:09:03.233 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-08-21T18:09:03.233 INFO:teuthology.orchestra.run.smithi169.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-08-21T18:09:03.233 INFO:teuthology.orchestra.run.smithi169.stdout:UBUNTU_CODENAME=jammy 2023-08-21T18:09:03.234 INFO:teuthology.provision.fog.smithi169:Deploy complete! 2023-08-21T18:09:03.234 DEBUG:teuthology.parallel:result is None 2023-08-21T18:09:03.412 INFO:teuthology.lock.ops:Checking smithi044.front.sepia.ceph.com 2023-08-21T18:09:03.413 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-21T18:09:03.441 INFO:teuthology.lock.ops:Checking smithi169.front.sepia.ceph.com 2023-08-21T18:09:03.441 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-21T18:09:03.458 INFO:teuthology.lock.ops:Updating [smithi044.front.sepia.ceph.com]: set os type and version on server 2023-08-21T18:09:03.458 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2023-08-21T18:09:03.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-21T18:09:03.533 DEBUG:teuthology.orchestra.run.smithi044:> true 2023-08-21T18:09:03.759 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2023-08-21T18:09:03.760 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2023-08-21T18:09:03.804 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2023-08-21T18:09:03.804 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2023-08-21T18:09:03.848 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-08-21T18:09:03.848 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="Ubuntu" 2023-08-21T18:09:03.849 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="22.04" 2023-08-21T18:09:03.849 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-08-21T18:09:03.849 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_CODENAME=jammy 2023-08-21T18:09:03.849 INFO:teuthology.orchestra.run.smithi044.stdout:ID=ubuntu 2023-08-21T18:09:03.849 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE=debian 2023-08-21T18:09:03.849 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://www.ubuntu.com/" 2023-08-21T18:09:03.849 INFO:teuthology.orchestra.run.smithi044.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-08-21T18:09:03.850 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-08-21T18:09:03.850 INFO:teuthology.orchestra.run.smithi044.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-08-21T18:09:03.850 INFO:teuthology.orchestra.run.smithi044.stdout:UBUNTU_CODENAME=jammy 2023-08-21T18:09:03.850 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2023-08-21T18:09:03.874 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: set os type and version on server 2023-08-21T18:09:03.874 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-08-21T18:09:03.874 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-21T18:09:03.948 DEBUG:teuthology.orchestra.run.smithi169:> true 2023-08-21T18:09:04.048 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2023-08-21T18:09:04.048 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2023-08-21T18:09:04.095 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2023-08-21T18:09:04.095 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2023-08-21T18:09:04.143 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-08-21T18:09:04.143 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="Ubuntu" 2023-08-21T18:09:04.143 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="22.04" 2023-08-21T18:09:04.144 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-08-21T18:09:04.144 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_CODENAME=jammy 2023-08-21T18:09:04.144 INFO:teuthology.orchestra.run.smithi169.stdout:ID=ubuntu 2023-08-21T18:09:04.144 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE=debian 2023-08-21T18:09:04.144 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://www.ubuntu.com/" 2023-08-21T18:09:04.144 INFO:teuthology.orchestra.run.smithi169.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-08-21T18:09:04.144 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-08-21T18:09:04.144 INFO:teuthology.orchestra.run.smithi169.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-08-21T18:09:04.145 INFO:teuthology.orchestra.run.smithi169.stdout:UBUNTU_CODENAME=jammy 2023-08-21T18:09:04.145 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2023-08-21T18:09:04.165 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:09:04.251 INFO:teuthology.dispatcher.supervisor:Running job 7374073 2023-08-21T18:09:04.251 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/virtualenv/bin/teuthology -v --owner scheduled_mkogan@teuthology --archive /home/teuthworker/archive/mkogan-2023-08-21_18:02:00-rgw:multisite-wip-rgw-t58917-distro-default-smithi/7374073 --name mkogan-2023-08-21_18:02:00-rgw:multisite-wip-rgw-t58917-distro-default-smithi --description rgw:multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{ubuntu_latest} tasks/test_multi} -- /home/teuthworker/archive/mkogan-2023-08-21_18:02:00-rgw:multisite-wip-rgw-t58917-distro-default-smithi/7374073/orig.config.yaml 2023-08-21T18:09:04.255 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/mkogan-2023-08-21_18:02:00-rgw:multisite-wip-rgw-t58917-distro-default-smithi/7374073 2023-08-21T18:09:04.255 INFO:teuthology.dispatcher.supervisor:Job PID: 14386 2023-08-21T18:09:04.256 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-08-21T18:11:04.356 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:13:04.521 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:15:04.677 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:17:04.818 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:19:04.988 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:21:05.065 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:23:05.141 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:25:05.213 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:27:05.332 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:29:05.553 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:31:05.948 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:33:06.120 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:35:06.272 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:37:06.437 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:39:06.604 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:41:06.768 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:43:06.929 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:45:07.085 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:47:07.257 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:49:07.428 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:51:07.498 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:53:07.658 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:55:07.823 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:57:07.943 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T18:59:08.054 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:01:08.216 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:03:08.329 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:05:08.497 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:07:08.653 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:09:09.033 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:11:09.195 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:13:09.367 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:15:09.507 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:17:09.677 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:19:09.828 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:21:10.036 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:23:10.266 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:25:10.420 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:27:10.833 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:29:11.032 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:31:11.197 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:33:11.353 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:35:11.523 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:37:11.772 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:39:12.330 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:41:12.726 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:43:13.829 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:45:14.364 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:47:14.633 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:49:14.780 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:51:14.942 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:53:15.036 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:55:15.205 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:57:15.431 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T19:59:15.593 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:01:15.766 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:03:16.164 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:05:16.329 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:07:16.494 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:09:16.713 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:11:16.914 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:13:17.430 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:15:17.773 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:17:18.020 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:19:18.189 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:21:18.373 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:23:18.502 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:25:18.661 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:27:18.829 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:29:19.007 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:31:19.172 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:33:19.341 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:35:19.509 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:37:19.741 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:39:19.916 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:41:20.081 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:43:20.246 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-21T20:43:20.314 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-08-21T20:43:20.497 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-08-21T20:43:20.498 INFO:teuthology.nuke:Checking targets against current locks 2023-08-21T20:43:20.549 DEBUG:teuthology.nuke:shortname: smithi044 2023-08-21T20:43:20.550 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-21T20:43:20.576 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mkogan-2023-08-21_18:02:00-rgw:multisite-wip-rgw-t58917-distro-default-smithi/7374073', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-21 18:03:00.009301', 'locked_by': 'scheduled_mkogan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHJ1qEq7nKfuN7OfAeEL3OOeMnlcJVDwwkbjqmNz5MCOBK2y+6HJA/p8cXsMRQqIBpqF1L5q/sk595uIJQFc8js='} 2023-08-21T20:43:20.579 DEBUG:teuthology.nuke:shortname: smithi169 2023-08-21T20:43:20.580 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-21T20:43:20.604 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mkogan-2023-08-21_18:02:00-rgw:multisite-wip-rgw-t58917-distro-default-smithi/7374073', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-21 18:03:00.007353', 'locked_by': 'scheduled_mkogan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAOCU19F0KloymOXruy7RxpfDhSrIEh2ZbFuH3Z0whDy'} 2023-08-21T20:43:20.636 INFO:teuthology.orchestra.console.smithi044:Power off 2023-08-21T20:43:20.637 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-21T20:43:20.660 DEBUG:teuthology.orchestra.console.smithi044:power off output: Chassis Power Control: Down/Off 2023-08-21T20:43:20.661 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-21T20:43:20.677 INFO:teuthology.orchestra.console.smithi169:Power off 2023-08-21T20:43:20.679 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-21T20:43:20.695 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2023-08-21T20:43:20.707 DEBUG:teuthology.orchestra.console.smithi169:power off output: Chassis Power Control: Down/Off 2023-08-21T20:43:20.708 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-21T20:43:20.729 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off 2023-08-21T20:43:20.796 INFO:teuthology.orchestra.console.smithi044:Power off completed 2023-08-21T20:43:20.831 INFO:teuthology.orchestra.console.smithi169:Power off completed 2023-08-21T20:43:20.946 INFO:teuthology.lock.ops:unlocked: smithi044.front.sepia.ceph.com 2023-08-21T20:43:20.981 INFO:teuthology.lock.ops:unlocked: smithi169.front.sepia.ceph.com