2023-07-11T19:01:42.622 INFO:root:teuthology version: 0.0.1.dev140+gb53f236 2023-07-11T19:01:42.623 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T19:01:42.623 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T19:01:42.775 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7febecd84d30>, signals=[15]) 2023-07-11T19:01:42.776 INFO:teuthology.lock.ops:Start node 'smithi090.front.sepia.ceph.com' reimaging 2023-07-11T19:01:42.777 INFO:teuthology.lock.ops:Updating [smithi090.front.sepia.ceph.com]: reset os type and version on server 2023-07-11T19:01:42.777 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2023-07-11T19:01:42.797 INFO:teuthology.lock.ops:Start node 'smithi174.front.sepia.ceph.com' reimaging 2023-07-11T19:01:42.798 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: reset os type and version on server 2023-07-11T19:01:42.798 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2023-07-11T19:01:42.801 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /host {"name": "smithi090"} 2023-07-11T19:01:42.821 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /host {"name": "smithi174"} 2023-07-11T19:01:42.874 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /image {"name": "smithi_ubuntu_22.04"} 2023-07-11T19:01:42.890 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /image {"name": "smithi_ubuntu_22.04"} 2023-07-11T19:01:42.928 DEBUG:teuthology.provision.fog.smithi090:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-07-11T19:01:42.928 DEBUG:teuthology.provision.fog.smithi090:FOG request: PUT /host/195 {"imageID": 6} 2023-07-11T19:01:42.943 DEBUG:teuthology.provision.fog.smithi174:Requesting image smithi_ubuntu_22.04 (ID 6) 2023-07-11T19:01:42.943 DEBUG:teuthology.provision.fog.smithi174:FOG request: PUT /host/279 {"imageID": 6} 2023-07-11T19:01:42.975 INFO:teuthology.provision.fog.smithi090:Scheduling deploy of ubuntu 22.04 2023-07-11T19:01:42.975 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:01:42.978 INFO:teuthology.provision.fog.smithi174:Scheduling deploy of ubuntu 22.04 2023-07-11T19:01:42.978 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:01:43.020 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /tasktype {"name": "deploy"} 2023-07-11T19:01:43.023 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /tasktype {"name": "deploy"} 2023-07-11T19:01:43.053 DEBUG:teuthology.provision.fog.smithi174:FOG request: POST /host/279/task {"taskTypeID": 1} 2023-07-11T19:01:43.059 DEBUG:teuthology.provision.fog.smithi090:FOG request: POST /host/195/task {"taskTypeID": 1} 2023-07-11T19:01:43.104 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:01:43.121 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:01:43.174 INFO:teuthology.orchestra.console:Power off smithi174 2023-07-11T19:01:43.175 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-11T19:01:43.191 INFO:teuthology.orchestra.console:Power off smithi090 2023-07-11T19:01:43.192 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-11T19:01:43.200 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T19:01:43.212 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T19:01:47.225 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T19:01:47.233 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T19:01:51.340 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T19:01:51.349 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T19:01:55.455 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T19:01:55.463 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T19:01:55.666 INFO:teuthology.orchestra.console:Power off for smithi174 completed 2023-07-11T19:01:55.673 INFO:teuthology.orchestra.console:Power off for smithi090 completed 2023-07-11T19:01:55.767 INFO:teuthology.orchestra.console:Power on smithi174 2023-07-11T19:01:55.768 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-11T19:01:55.775 INFO:teuthology.orchestra.console:Power on smithi090 2023-07-11T19:01:55.776 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-11T19:01:55.788 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T19:01:55.795 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T19:01:59.812 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T19:01:59.819 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T19:02:00.022 INFO:teuthology.orchestra.console:Power on for smithi174 completed 2023-07-11T19:02:00.029 INFO:teuthology.orchestra.console:Power on for smithi090 completed 2023-07-11T19:02:00.124 INFO:teuthology.provision.fog.smithi174:Waiting for deploy to finish 2023-07-11T19:02:00.124 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:02:00.131 INFO:teuthology.provision.fog.smithi090:Waiting for deploy to finish 2023-07-11T19:02:00.131 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:02:15.215 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:02:15.217 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:02:30.316 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:02:30.317 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:02:45.415 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:02:45.417 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:03:00.495 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:03:00.496 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:03:15.584 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:03:15.586 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:03:30.672 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:03:30.673 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:03:45.773 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:03:45.774 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:04:00.865 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:04:00.867 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:04:15.960 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:04:15.961 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:04:31.048 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:04:31.049 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:04:46.122 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:04:46.123 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:05:01.202 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:05:01.203 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:05:16.265 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:05:16.267 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:05:31.336 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:05:31.338 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:05:46.403 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:05:46.404 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:06:01.452 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:06:01.458 DEBUG:teuthology.provision.fog.smithi174:FOG request: GET /task/active None 2023-07-11T19:06:01.521 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:06:04.592 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-07-11T19:06:10.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:06:13.684 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-07-11T19:06:16.501 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None 2023-07-11T19:06:16.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:06:19.688 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:06:22.768 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-07-11T19:06:28.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:06:31.824 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-07-11T19:06:34.996 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2023-07-11T19:06:37.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:06:40.880 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-07-11T19:06:40.997 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:06:44.048 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2023-07-11T19:06:46.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:06:49.936 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-07-11T19:06:50.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:06:53.104 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2023-07-11T19:06:55.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:06:58.992 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-07-11T19:06:59.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:07:02.160 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2023-07-11T19:07:04.996 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:07:08.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:07:11.216 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2023-07-11T19:07:17.220 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:07:20.272 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2023-07-11T19:07:26.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:07:29.328 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2023-07-11T19:07:35.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:07:36.436 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-07-11T19:07:42.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:07:42.442 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-07-11T19:07:48.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:07:48.450 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-07-11T19:07:54.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:07:54.458 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2023-07-11T19:08:00.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:08:00.967 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-11T19:08:00.968 DEBUG:teuthology.orchestra.run.smithi174:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-11T19:08:01.771 INFO:teuthology.provision.fog.smithi174:Node is ready 2023-07-11T19:08:01.771 DEBUG:teuthology.orchestra.run.smithi174:> hostname 2023-07-11T19:08:01.818 INFO:teuthology.orchestra.run.smithi174.stdout:smithi174.front.sepia.ceph.com 2023-07-11T19:08:01.818 DEBUG:teuthology.orchestra.run.smithi174:> grep smithi174.front.sepia.ceph.com /etc/hosts 2023-07-11T19:08:01.867 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.15.174 smithi174.front.sepia.ceph.com smithi174 2023-07-11T19:08:01.867 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname smithi174 2023-07-11T19:08:02.033 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/smithi174.front.sepia.ceph.com/smithi174/g' /etc/hosts 2023-07-11T19:08:02.046 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/172.21.15.174/172.21.15.174/g' /etc/hosts 2023-07-11T19:08:02.100 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/smithi174.front.sepia.ceph.com/smithi174/g' /etc/hostname 2023-07-11T19:08:02.152 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname smithi174 2023-07-11T19:08:02.205 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2023-07-11T19:08:02.253 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-07-11T19:08:02.253 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="Ubuntu" 2023-07-11T19:08:02.253 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="22.04" 2023-07-11T19:08:02.254 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-07-11T19:08:02.254 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_CODENAME=jammy 2023-07-11T19:08:02.254 INFO:teuthology.orchestra.run.smithi174.stdout:ID=ubuntu 2023-07-11T19:08:02.254 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE=debian 2023-07-11T19:08:02.254 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://www.ubuntu.com/" 2023-07-11T19:08:02.255 INFO:teuthology.orchestra.run.smithi174.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-07-11T19:08:02.255 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-07-11T19:08:02.255 INFO:teuthology.orchestra.run.smithi174.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-07-11T19:08:02.255 INFO:teuthology.orchestra.run.smithi174.stdout:UBUNTU_CODENAME=jammy 2023-07-11T19:08:02.256 INFO:teuthology.provision.fog.smithi174:Deploy complete! 2023-07-11T19:08:02.257 DEBUG:teuthology.parallel:result is None 2023-07-11T19:08:35.333 WARNING:teuthology.provision.fog.smithi090:timed out 2023-07-11T19:08:41.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:08:41.645 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-11T19:08:41.645 DEBUG:teuthology.orchestra.run.smithi090:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-11T19:08:42.201 INFO:teuthology.provision.fog.smithi090:Node is ready 2023-07-11T19:08:42.201 DEBUG:teuthology.orchestra.run.smithi090:> hostname 2023-07-11T19:08:42.250 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090.front.sepia.ceph.com 2023-07-11T19:08:42.251 DEBUG:teuthology.orchestra.run.smithi090:> grep smithi090.front.sepia.ceph.com /etc/hosts 2023-07-11T19:08:42.297 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90 smithi090.front.sepia.ceph.com smithi090 2023-07-11T19:08:42.298 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname smithi090 2023-07-11T19:08:42.447 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/smithi090.front.sepia.ceph.com/smithi090/g' /etc/hosts 2023-07-11T19:08:42.460 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/172.21.15.90/172.21.15.90/g' /etc/hosts 2023-07-11T19:08:42.513 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/smithi090.front.sepia.ceph.com/smithi090/g' /etc/hostname 2023-07-11T19:08:42.573 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname smithi090 2023-07-11T19:08:42.629 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2023-07-11T19:08:42.681 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-07-11T19:08:42.681 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="Ubuntu" 2023-07-11T19:08:42.681 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="22.04" 2023-07-11T19:08:42.681 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-07-11T19:08:42.681 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_CODENAME=jammy 2023-07-11T19:08:42.682 INFO:teuthology.orchestra.run.smithi090.stdout:ID=ubuntu 2023-07-11T19:08:42.682 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE=debian 2023-07-11T19:08:42.682 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://www.ubuntu.com/" 2023-07-11T19:08:42.682 INFO:teuthology.orchestra.run.smithi090.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-07-11T19:08:42.682 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-07-11T19:08:42.682 INFO:teuthology.orchestra.run.smithi090.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-07-11T19:08:42.682 INFO:teuthology.orchestra.run.smithi090.stdout:UBUNTU_CODENAME=jammy 2023-07-11T19:08:42.683 INFO:teuthology.provision.fog.smithi090:Deploy complete! 2023-07-11T19:08:42.683 DEBUG:teuthology.parallel:result is None 2023-07-11T19:08:42.845 INFO:teuthology.lock.ops:Checking smithi090.front.sepia.ceph.com 2023-07-11T19:08:42.846 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-11T19:08:42.874 INFO:teuthology.lock.ops:Checking smithi174.front.sepia.ceph.com 2023-07-11T19:08:42.875 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-11T19:08:42.892 INFO:teuthology.lock.ops:Updating [smithi090.front.sepia.ceph.com]: set os type and version on server 2023-07-11T19:08:42.892 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2023-07-11T19:08:42.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:08:42.970 DEBUG:teuthology.orchestra.run.smithi090:> true 2023-07-11T19:08:43.072 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2023-07-11T19:08:43.073 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2023-07-11T19:08:43.117 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2023-07-11T19:08:43.117 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2023-07-11T19:08:43.165 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-07-11T19:08:43.165 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="Ubuntu" 2023-07-11T19:08:43.166 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="22.04" 2023-07-11T19:08:43.166 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-07-11T19:08:43.166 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_CODENAME=jammy 2023-07-11T19:08:43.166 INFO:teuthology.orchestra.run.smithi090.stdout:ID=ubuntu 2023-07-11T19:08:43.166 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE=debian 2023-07-11T19:08:43.166 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://www.ubuntu.com/" 2023-07-11T19:08:43.166 INFO:teuthology.orchestra.run.smithi090.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-07-11T19:08:43.167 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-07-11T19:08:43.167 INFO:teuthology.orchestra.run.smithi090.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-07-11T19:08:43.167 INFO:teuthology.orchestra.run.smithi090.stdout:UBUNTU_CODENAME=jammy 2023-07-11T19:08:43.167 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2023-07-11T19:08:43.191 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: set os type and version on server 2023-07-11T19:08:43.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2023-07-11T19:08:43.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-11T19:08:43.265 DEBUG:teuthology.orchestra.run.smithi174:> true 2023-07-11T19:08:43.384 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2023-07-11T19:08:43.384 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2023-07-11T19:08:43.432 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2023-07-11T19:08:43.432 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2023-07-11T19:08:43.477 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-07-11T19:08:43.477 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="Ubuntu" 2023-07-11T19:08:43.477 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="22.04" 2023-07-11T19:08:43.477 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-07-11T19:08:43.478 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_CODENAME=jammy 2023-07-11T19:08:43.478 INFO:teuthology.orchestra.run.smithi174.stdout:ID=ubuntu 2023-07-11T19:08:43.478 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE=debian 2023-07-11T19:08:43.479 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://www.ubuntu.com/" 2023-07-11T19:08:43.479 INFO:teuthology.orchestra.run.smithi174.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-07-11T19:08:43.479 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-07-11T19:08:43.480 INFO:teuthology.orchestra.run.smithi174.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-07-11T19:08:43.480 INFO:teuthology.orchestra.run.smithi174.stdout:UBUNTU_CODENAME=jammy 2023-07-11T19:08:43.480 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2023-07-11T19:08:43.508 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T19:08:43.509 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T19:08:43.612 INFO:teuthology.dispatcher.supervisor:Running job 7334119 2023-07-11T19:08:43.612 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/virtualenv/bin/teuthology -v --owner scheduled_choffman@teuthology --archive /home/teuthworker/archive/choffman-2023-07-11_19:00:51-rbd:mirror-wip-choffman-61707-distro-default-smithi/7334119 --name choffman-2023-07-11_19:00:51-rbd:mirror-wip-choffman-61707-distro-default-smithi --description rbd:mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-exclusive-lock} -- /home/teuthworker/archive/choffman-2023-07-11_19:00:51-rbd:mirror-wip-choffman-61707-distro-default-smithi/7334119/orig.config.yaml 2023-07-11T19:08:43.616 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/choffman-2023-07-11_19:00:51-rbd:mirror-wip-choffman-61707-distro-default-smithi/7334119 2023-07-11T19:08:43.616 INFO:teuthology.dispatcher.supervisor:Job PID: 31103 2023-07-11T19:08:43.617 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-11T19:10:43.676 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T19:10:43.678 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T19:12:43.849 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T19:12:43.850 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T19:14:44.025 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T19:14:44.026 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T19:16:44.197 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T19:16:44.199 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T19:18:44.361 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T19:18:44.363 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T19:20:44.546 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T19:20:44.547 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T19:22:44.721 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T19:22:44.722 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T19:24:44.890 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T19:24:44.891 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T19:26:45.058 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-11T19:26:45.059 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-11T19:26:45.131 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-07-11T19:26:45.249 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-07-11T19:26:45.249 INFO:teuthology.nuke:Checking targets against current locks 2023-07-11T19:26:45.305 DEBUG:teuthology.nuke:shortname: smithi090 2023-07-11T19:26:45.306 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-11T19:26:45.330 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-07-11_19:00:51-rbd:mirror-wip-choffman-61707-distro-default-smithi/7334119', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-11 19:01:41.801532', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIL5yDBJZrLOZGoLItUWIo/T7pLXGpTGeGB1C3FvXORuI'} 2023-07-11T19:26:45.334 DEBUG:teuthology.nuke:shortname: smithi174 2023-07-11T19:26:45.335 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-11T19:26:45.356 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-07-11_19:00:51-rbd:mirror-wip-choffman-61707-distro-default-smithi/7334119', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-11 19:01:41.803106', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLV4aQmDCHpwZaWtBuTapl51PWCyw01reCfD1S5Dx2g9F3Nb5PgtvW841N9eODqZL+SbOKUYgssGWj6ugsmY5aQ='} 2023-07-11T19:26:45.382 INFO:teuthology.orchestra.console:Power off smithi090 2023-07-11T19:26:45.382 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-11T19:26:45.404 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T19:26:45.426 INFO:teuthology.orchestra.console:Power off smithi174 2023-07-11T19:26:45.427 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-11T19:26:45.450 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-11T19:26:45.539 INFO:teuthology.orchestra.console:Power off for smithi090 completed 2023-07-11T19:26:45.574 INFO:teuthology.orchestra.console:Power off for smithi174 completed 2023-07-11T19:26:45.696 INFO:teuthology.lock.ops:unlocked: smithi090.front.sepia.ceph.com 2023-07-11T19:26:45.730 INFO:teuthology.lock.ops:unlocked: smithi174.front.sepia.ceph.com