2023-07-08T06:32:00.568 INFO:root:teuthology version: 0.0.1.dev140+gb53f236 2023-07-08T06:32:00.569 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-08T06:32:00.570 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-08T06:32:00.704 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f86bc6a98b0>, signals=[15]) 2023-07-08T06:32:00.706 INFO:teuthology.lock.ops:Start node 'smithi133.front.sepia.ceph.com' reimaging 2023-07-08T06:32:00.707 INFO:teuthology.lock.ops:Updating [smithi133.front.sepia.ceph.com]: reset os type and version on server 2023-07-08T06:32:00.707 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2023-07-08T06:32:00.729 INFO:teuthology.lock.ops:Start node 'smithi202.front.sepia.ceph.com' reimaging 2023-07-08T06:32:00.729 INFO:teuthology.lock.ops:Updating [smithi202.front.sepia.ceph.com]: reset os type and version on server 2023-07-08T06:32:00.730 INFO:teuthology.lock.ops:Updating smithi202.front.sepia.ceph.com on lock server 2023-07-08T06:32:00.732 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /host {"name": "smithi133"} 2023-07-08T06:32:00.753 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /host {"name": "smithi202"} 2023-07-08T06:32:00.857 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-08T06:32:00.862 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-08T06:32:00.900 DEBUG:teuthology.provision.fog.smithi133:Requesting image smithi_centos_8.stream (ID 3) 2023-07-08T06:32:00.901 DEBUG:teuthology.provision.fog.smithi133:FOG request: PUT /host/238 {"imageID": 3} 2023-07-08T06:32:00.907 DEBUG:teuthology.provision.fog.smithi202:Requesting image smithi_centos_8.stream (ID 3) 2023-07-08T06:32:00.907 DEBUG:teuthology.provision.fog.smithi202:FOG request: PUT /host/306 {"imageID": 3} 2023-07-08T06:32:00.936 INFO:teuthology.provision.fog.smithi133:Scheduling deploy of centos 8.stream 2023-07-08T06:32:00.936 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:32:00.961 INFO:teuthology.provision.fog.smithi202:Scheduling deploy of centos 8.stream 2023-07-08T06:32:00.961 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:32:01.134 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /tasktype {"name": "deploy"} 2023-07-08T06:32:01.177 DEBUG:teuthology.provision.fog.smithi133:FOG request: POST /host/238/task {"taskTypeID": 1} 2023-07-08T06:32:01.192 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /tasktype {"name": "deploy"} 2023-07-08T06:32:01.223 DEBUG:teuthology.provision.fog.smithi202:FOG request: POST /host/306/task {"taskTypeID": 1} 2023-07-08T06:32:01.230 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:32:01.280 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:32:01.484 INFO:teuthology.orchestra.console:Power off smithi133 2023-07-08T06:32:01.485 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-08T06:32:01.505 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-08T06:32:01.533 INFO:teuthology.orchestra.console:Power off smithi202 2023-07-08T06:32:01.534 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-08T06:32:01.556 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-08T06:32:01.626 INFO:teuthology.orchestra.console:Power off for smithi133 completed 2023-07-08T06:32:01.727 INFO:teuthology.orchestra.console:Power on smithi133 2023-07-08T06:32:01.727 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-08T06:32:01.748 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-08T06:32:05.583 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-08T06:32:05.768 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-08T06:32:09.699 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-08T06:32:09.879 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-08T06:32:09.908 INFO:teuthology.orchestra.console:Power off for smithi202 completed 2023-07-08T06:32:10.010 INFO:teuthology.orchestra.console:Power on smithi202 2023-07-08T06:32:10.010 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-08T06:32:10.033 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-08T06:32:10.088 INFO:teuthology.orchestra.console:Power on for smithi133 completed 2023-07-08T06:32:10.189 INFO:teuthology.provision.fog.smithi133:Waiting for deploy to finish 2023-07-08T06:32:10.190 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:32:14.059 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-08T06:32:18.172 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-08T06:32:18.464 INFO:teuthology.orchestra.console:Power on for smithi202 completed 2023-07-08T06:32:18.565 INFO:teuthology.provision.fog.smithi202:Waiting for deploy to finish 2023-07-08T06:32:18.566 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:32:25.448 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:32:33.805 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:32:40.720 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:32:49.093 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:32:55.968 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:33:04.332 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:33:11.216 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:33:19.631 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:33:26.504 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:33:34.929 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:33:41.792 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:33:50.310 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:33:58.008 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:34:05.756 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:34:15.259 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:34:21.229 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:34:30.729 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:34:39.680 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:34:47.592 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:34:56.796 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:35:03.939 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:35:13.544 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:35:20.892 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:35:30.508 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:35:37.468 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:35:48.516 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:35:54.824 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:36:04.026 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:36:11.724 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:36:19.872 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:36:28.581 DEBUG:teuthology.provision.fog.smithi133:FOG request: GET /task/active None 2023-07-08T06:36:30.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T06:36:33.168 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133 2023-07-08T06:36:37.572 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:36:39.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T06:36:42.228 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133 2023-07-08T06:36:48.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T06:36:51.316 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133 2023-07-08T06:36:54.568 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:36:57.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T06:37:00.368 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133 2023-07-08T06:37:06.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T06:37:09.456 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133 2023-07-08T06:37:11.218 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-07-08T06:37:12.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T06:37:15.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T06:37:18.512 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133 2023-07-08T06:37:24.519 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T06:37:31.376 WARNING:teuthology.provision.fog.smithi202:[Errno None] Unable to connect to port 22 on 172.21.15.202 2023-07-08T06:37:37.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T06:37:40.436 WARNING:teuthology.provision.fog.smithi202:[Errno None] Unable to connect to port 22 on 172.21.15.202 2023-07-08T06:37:46.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T06:37:49.520 WARNING:teuthology.provision.fog.smithi202:[Errno None] Unable to connect to port 22 on 172.21.15.202 2023-07-08T06:37:55.527 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T06:37:58.608 WARNING:teuthology.provision.fog.smithi202:[Errno None] Unable to connect to port 22 on 172.21.15.202 2023-07-08T06:38:04.612 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T06:38:07.664 WARNING:teuthology.provision.fog.smithi202:[Errno None] Unable to connect to port 22 on 172.21.15.202 2023-07-08T06:38:13.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T06:38:24.528 WARNING:teuthology.provision.fog.smithi133:timed out 2023-07-08T06:38:30.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T06:38:30.745 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-08T06:38:30.746 DEBUG:teuthology.orchestra.run.smithi133:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-08T06:38:30.988 INFO:teuthology.provision.fog.smithi133:Node is ready 2023-07-08T06:38:30.989 DEBUG:teuthology.orchestra.run.smithi133:> hostname 2023-07-08T06:38:31.007 INFO:teuthology.orchestra.run.smithi133.stdout:smithi133.front.sepia.ceph.com 2023-07-08T06:38:31.008 DEBUG:teuthology.orchestra.run.smithi133:> grep smithi133.front.sepia.ceph.com /etc/hosts 2023-07-08T06:38:31.066 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133 smithi133.front.sepia.ceph.com smithi133 2023-07-08T06:38:31.067 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname smithi133 2023-07-08T06:38:31.156 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i -e 's/smithi133.front.sepia.ceph.com/smithi133/g' /etc/hosts 2023-07-08T06:38:31.184 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i -e 's/172.21.15.133/172.21.15.133/g' /etc/hosts 2023-07-08T06:38:31.252 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i -e 's/smithi133.front.sepia.ceph.com/smithi133/g' /etc/hostname 2023-07-08T06:38:31.319 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname smithi133 2023-07-08T06:38:31.385 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2023-07-08T06:38:31.442 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2023-07-08T06:38:31.442 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="8" 2023-07-08T06:38:31.443 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2023-07-08T06:38:31.443 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2023-07-08T06:38:31.443 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="8" 2023-07-08T06:38:31.443 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el8" 2023-07-08T06:38:31.443 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-08T06:38:31.443 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2023-07-08T06:38:31.443 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-08T06:38:31.444 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2023-07-08T06:38:31.444 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-08T06:38:31.444 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-08T06:38:31.444 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-08T06:38:31.445 INFO:teuthology.provision.fog.smithi133:Deploy complete! 2023-07-08T06:38:31.446 DEBUG:teuthology.parallel:result is None 2023-07-08T06:39:13.712 WARNING:teuthology.provision.fog.smithi202:timed out 2023-07-08T06:39:19.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T06:39:19.969 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-08T06:39:19.969 DEBUG:teuthology.orchestra.run.smithi202:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-08T06:39:20.230 INFO:teuthology.provision.fog.smithi202:Node is ready 2023-07-08T06:39:20.231 DEBUG:teuthology.orchestra.run.smithi202:> hostname 2023-07-08T06:39:20.252 INFO:teuthology.orchestra.run.smithi202.stdout:smithi202.front.sepia.ceph.com 2023-07-08T06:39:20.252 DEBUG:teuthology.orchestra.run.smithi202:> grep smithi202.front.sepia.ceph.com /etc/hosts 2023-07-08T06:39:20.314 INFO:teuthology.orchestra.run.smithi202.stdout:172.21.15.202 smithi202.front.sepia.ceph.com smithi202 2023-07-08T06:39:20.315 DEBUG:teuthology.orchestra.run.smithi202:> sudo hostname smithi202 2023-07-08T06:39:20.415 DEBUG:teuthology.orchestra.run.smithi202:> sudo sed -i -e 's/smithi202.front.sepia.ceph.com/smithi202/g' /etc/hosts 2023-07-08T06:39:20.447 DEBUG:teuthology.orchestra.run.smithi202:> sudo sed -i -e 's/172.21.15.202/172.21.15.202/g' /etc/hosts 2023-07-08T06:39:20.517 DEBUG:teuthology.orchestra.run.smithi202:> sudo sed -i -e 's/smithi202.front.sepia.ceph.com/smithi202/g' /etc/hostname 2023-07-08T06:39:20.586 DEBUG:teuthology.orchestra.run.smithi202:> sudo hostname smithi202 2023-07-08T06:39:20.654 DEBUG:teuthology.orchestra.run.smithi202:> cat /etc/os-release 2023-07-08T06:39:20.712 INFO:teuthology.orchestra.run.smithi202.stdout:NAME="CentOS Stream" 2023-07-08T06:39:20.713 INFO:teuthology.orchestra.run.smithi202.stdout:VERSION="8" 2023-07-08T06:39:20.713 INFO:teuthology.orchestra.run.smithi202.stdout:ID="centos" 2023-07-08T06:39:20.713 INFO:teuthology.orchestra.run.smithi202.stdout:ID_LIKE="rhel fedora" 2023-07-08T06:39:20.713 INFO:teuthology.orchestra.run.smithi202.stdout:VERSION_ID="8" 2023-07-08T06:39:20.713 INFO:teuthology.orchestra.run.smithi202.stdout:PLATFORM_ID="platform:el8" 2023-07-08T06:39:20.713 INFO:teuthology.orchestra.run.smithi202.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-08T06:39:20.713 INFO:teuthology.orchestra.run.smithi202.stdout:ANSI_COLOR="0;31" 2023-07-08T06:39:20.714 INFO:teuthology.orchestra.run.smithi202.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-08T06:39:20.714 INFO:teuthology.orchestra.run.smithi202.stdout:HOME_URL="https://centos.org/" 2023-07-08T06:39:20.714 INFO:teuthology.orchestra.run.smithi202.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-08T06:39:20.714 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-08T06:39:20.714 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-08T06:39:20.714 INFO:teuthology.provision.fog.smithi202:Deploy complete! 2023-07-08T06:39:20.715 DEBUG:teuthology.parallel:result is None 2023-07-08T06:39:20.894 INFO:teuthology.lock.ops:Checking smithi133.front.sepia.ceph.com 2023-07-08T06:39:20.894 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-08T06:39:20.925 INFO:teuthology.lock.ops:Checking smithi202.front.sepia.ceph.com 2023-07-08T06:39:20.926 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-08T06:39:20.943 INFO:teuthology.lock.ops:Updating [smithi133.front.sepia.ceph.com]: set os type and version on server 2023-07-08T06:39:20.943 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-07-08T06:39:20.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T06:39:21.009 DEBUG:teuthology.orchestra.run.smithi133:> true 2023-07-08T06:39:21.146 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-07-08T06:39:21.146 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2023-07-08T06:39:21.204 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2023-07-08T06:39:21.204 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2023-07-08T06:39:21.262 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2023-07-08T06:39:21.262 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="8" 2023-07-08T06:39:21.262 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2023-07-08T06:39:21.262 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2023-07-08T06:39:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="8" 2023-07-08T06:39:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el8" 2023-07-08T06:39:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-08T06:39:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2023-07-08T06:39:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-08T06:39:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2023-07-08T06:39:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-08T06:39:21.264 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-08T06:39:21.264 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-08T06:39:21.264 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2023-07-08T06:39:21.284 INFO:teuthology.lock.ops:Updating [smithi202.front.sepia.ceph.com]: set os type and version on server 2023-07-08T06:39:21.284 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi202.front.sepia.ceph.com' 2023-07-08T06:39:21.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-08T06:39:21.352 DEBUG:teuthology.orchestra.run.smithi202:> true 2023-07-08T06:39:21.427 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi202.front.sepia.ceph.com' 2023-07-08T06:39:21.427 DEBUG:teuthology.orchestra.run.smithi202:> uname -m 2023-07-08T06:39:21.486 INFO:teuthology.orchestra.run.smithi202.stdout:x86_64 2023-07-08T06:39:21.487 DEBUG:teuthology.orchestra.run.smithi202:> cat /etc/os-release 2023-07-08T06:39:21.545 INFO:teuthology.orchestra.run.smithi202.stdout:NAME="CentOS Stream" 2023-07-08T06:39:21.545 INFO:teuthology.orchestra.run.smithi202.stdout:VERSION="8" 2023-07-08T06:39:21.545 INFO:teuthology.orchestra.run.smithi202.stdout:ID="centos" 2023-07-08T06:39:21.545 INFO:teuthology.orchestra.run.smithi202.stdout:ID_LIKE="rhel fedora" 2023-07-08T06:39:21.545 INFO:teuthology.orchestra.run.smithi202.stdout:VERSION_ID="8" 2023-07-08T06:39:21.546 INFO:teuthology.orchestra.run.smithi202.stdout:PLATFORM_ID="platform:el8" 2023-07-08T06:39:21.546 INFO:teuthology.orchestra.run.smithi202.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-08T06:39:21.546 INFO:teuthology.orchestra.run.smithi202.stdout:ANSI_COLOR="0;31" 2023-07-08T06:39:21.546 INFO:teuthology.orchestra.run.smithi202.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-08T06:39:21.546 INFO:teuthology.orchestra.run.smithi202.stdout:HOME_URL="https://centos.org/" 2023-07-08T06:39:21.546 INFO:teuthology.orchestra.run.smithi202.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-08T06:39:21.546 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-08T06:39:21.547 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-08T06:39:21.547 INFO:teuthology.lock.ops:Updating smithi202.front.sepia.ceph.com on lock server 2023-07-08T06:39:21.576 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-08T06:39:21.576 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-08T06:39:21.670 INFO:teuthology.dispatcher.supervisor:Running job 7328977 2023-07-08T06:39:21.670 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2023-07-08_06:30:43-orch:cephadm-wip-adk4-testing-2023-07-07-2139-distro-default-smithi/7328977 --name adking-2023-07-08_06:30:43-orch:cephadm-wip-adk4-testing-2023-07-07-2139-distro-default-smithi --description orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/adking-2023-07-08_06:30:43-orch:cephadm-wip-adk4-testing-2023-07-07-2139-distro-default-smithi/7328977/orig.config.yaml 2023-07-08T06:39:21.673 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2023-07-08_06:30:43-orch:cephadm-wip-adk4-testing-2023-07-07-2139-distro-default-smithi/7328977 2023-07-08T06:39:21.674 INFO:teuthology.dispatcher.supervisor:Job PID: 26642 2023-07-08T06:39:21.674 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-08T06:41:21.680 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-08T06:41:21.681 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-08T06:43:21.848 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-08T06:43:21.849 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-08T06:45:22.021 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-08T06:45:22.021 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-08T06:47:22.195 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-08T06:47:22.196 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-08T06:49:22.381 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-08T06:49:22.382 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-08T06:51:22.640 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-08T06:51:22.640 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-08T06:53:22.916 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-08T06:53:22.917 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-08T06:55:23.221 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-08T06:55:23.222 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-08T06:57:23.421 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-08T06:57:23.421 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-08T06:57:23.519 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-07-08T06:57:23.726 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-07-08T06:57:23.727 INFO:teuthology.nuke:Checking targets against current locks 2023-07-08T06:57:23.808 DEBUG:teuthology.nuke:shortname: smithi133 2023-07-08T06:57:23.809 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-08T06:57:23.843 DEBUG:teuthology.nuke:shortname: smithi202 2023-07-08T06:57:23.844 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-08T06:57:23.853 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2023-07-08_06:30:43-orch:cephadm-wip-adk4-testing-2023-07-07-2139-distro-default-smithi/7328977', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-08 06:31:59.784827', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuXcUiEQkE/Q9BUSxgiaf27Lp6NRxU5nFyFiYheu4gK3lIJNcc5htxD12u59AEY1D/n4497uR2voDd32VQwvNfyW4Y7uF731HjLp7tibtWADQncCMkc4IEF+ShdWe6bo8HiypSEBnIaka1BRbzzG9PH0ppcj/6cMZ/BVwhSuJRvpsnF9DMdJ+5CVwaRk81+d5CA6tuZ+bfdGlWQdvZjQM1By7xFeSMsMvXSZaQSU/0AhBX8+Js5EfOYTwwG23OPyV+DmEk0S5oaSZ5RnvoCc7Rs/k9oZKsmB+lIWz3hXQfyOSTyPwtjU6nBz8rqOwAYCGCAh59ZMrDgbEej11/QSl5v6CcfjDX5JgcmBa79Kd1MQYK3brM9LKUeLnTPSbiFmjEfMJrshTOcnXKM+0L3I2f30mGFPpfTjUGzkpEOwo3Fd8TeENP8k8lSzX+HZ+WHeWgZix2XMp8CZ2Kzh4uss97AVHcs0u+7+IzltQfpwMJE+xrvYU/2wrNYx5GdgnSE+M='} 2023-07-08T06:57:23.891 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi202.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2023-07-08_06:30:43-orch:cephadm-wip-adk4-testing-2023-07-07-2139-distro-default-smithi/7328977', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-08 06:31:59.782890', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDccQCByldsBXSh1IKc/9iIbS8kns3ydqfQE7gbYvWIATCA/8Vy6DTVHDKa3wAWPlDdfGcctd/4lF1OByLDbaTrgNRkDOJGIKacEuWcajM+tBiACmvNSiy8JVGCXsIKJK98OQGYImheuc75b+HL3vOLUvZxdraN2/5auNs2ziIUzjfVqSadIStI+B5Lc//RinqLvUslrHny2XVl5rNox4xtP/DzpF4q++7PhnnTnfOSMw1AOnjpYHAr8lSK2i5ftP3vk0k0K2tLD6BUiMPN/EVhXgZKsQ9jhDYQ/LIwu9Cv5KpvAG8tK024zhXFty4SDos3ZWCBE1S2COpzjQDdlVk9kE/Ne5paJmejk/UUB97dqqphDMhefPuXQPgHUWXyzhCEgyLuEw3iSy+nHCkTJMy3J4LY4JLwHnOPRb2RtPWHjR0izEDa109gzg65HWKh8nBdwI08YLcXLy4fYbgPnZheyYURpjv8SaZlGpOmEgcZWMkqvZ12MHX3I3D+qYiHF4M='} 2023-07-08T06:57:23.937 INFO:teuthology.orchestra.console:Power off smithi133 2023-07-08T06:57:23.938 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-08T06:57:23.960 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-08T06:57:23.991 INFO:teuthology.orchestra.console:Power off smithi202 2023-07-08T06:57:23.992 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-08T06:57:24.017 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-08T06:57:24.109 INFO:teuthology.orchestra.console:Power off for smithi133 completed 2023-07-08T06:57:24.148 INFO:teuthology.orchestra.console:Power off for smithi202 completed 2023-07-08T06:57:24.283 INFO:teuthology.lock.ops:unlocked: smithi133.front.sepia.ceph.com 2023-07-08T06:57:24.315 INFO:teuthology.lock.ops:unlocked: smithi202.front.sepia.ceph.com