2023-12-06T03:45:25.261 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-06T03:45:25.261 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T03:45:25.389 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd12110e550>, signals=[15]) 2023-12-06T03:45:25.389 INFO:teuthology.lock.ops:Start node 'smithi077.front.sepia.ceph.com' reimaging 2023-12-06T03:45:25.389 INFO:teuthology.lock.ops:Updating [smithi077.front.sepia.ceph.com]: reset os type and version on server 2023-12-06T03:45:25.389 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2023-12-06T03:45:25.409 INFO:teuthology.lock.ops:Start node 'smithi167.front.sepia.ceph.com' reimaging 2023-12-06T03:45:25.409 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: reset os type and version on server 2023-12-06T03:45:25.410 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2023-12-06T03:45:25.549 DEBUG:teuthology.provision.fog.smithi077:Requesting image smithi_centos_8.stream (ID 3) 2023-12-06T03:45:25.554 DEBUG:teuthology.provision.fog.smithi167:Requesting image smithi_centos_8.stream (ID 3) 2023-12-06T03:45:25.594 INFO:teuthology.provision.fog.smithi077:Scheduling deploy of centos 8.stream 2023-12-06T03:45:25.606 INFO:teuthology.provision.fog.smithi167:Scheduling deploy of centos 8.stream 2023-12-06T03:45:25.902 INFO:teuthology.orchestra.console.smithi077:Power off 2023-12-06T03:45:25.903 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-06T03:45:25.918 INFO:teuthology.orchestra.console.smithi167:Power off 2023-12-06T03:45:25.918 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-06T03:45:25.926 DEBUG:teuthology.orchestra.console.smithi077:power off output: Chassis Power Control: Down/Off 2023-12-06T03:45:25.927 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:45:25.940 DEBUG:teuthology.orchestra.console.smithi167:power off output: Chassis Power Control: Down/Off 2023-12-06T03:45:25.941 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:45:25.950 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is on 2023-12-06T03:45:25.963 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2023-12-06T03:45:29.954 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:45:29.964 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:45:30.064 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is on 2023-12-06T03:45:30.072 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2023-12-06T03:45:34.066 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:45:34.074 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:45:34.174 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is off 2023-12-06T03:45:34.182 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2023-12-06T03:45:34.276 INFO:teuthology.orchestra.console.smithi077:Power off completed 2023-12-06T03:45:34.377 INFO:teuthology.orchestra.console.smithi077:Power on 2023-12-06T03:45:34.377 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-06T03:45:34.398 DEBUG:teuthology.orchestra.console.smithi077:power on output: Chassis Power Control: Up/On 2023-12-06T03:45:34.398 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:45:34.419 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is off 2023-12-06T03:45:38.186 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:45:38.295 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off 2023-12-06T03:45:38.396 INFO:teuthology.orchestra.console.smithi167:Power off completed 2023-12-06T03:45:38.420 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:45:38.498 INFO:teuthology.orchestra.console.smithi167:Power on 2023-12-06T03:45:38.498 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-06T03:45:38.518 DEBUG:teuthology.orchestra.console.smithi167:power on output: Chassis Power Control: Up/On 2023-12-06T03:45:38.519 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:45:38.528 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is on 2023-12-06T03:45:38.541 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off 2023-12-06T03:45:38.629 INFO:teuthology.orchestra.console.smithi077:Power on completed 2023-12-06T03:45:38.730 INFO:teuthology.provision.fog.smithi077:Waiting for deploy to finish 2023-12-06T03:45:42.542 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:45:42.651 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2023-12-06T03:45:42.751 INFO:teuthology.orchestra.console.smithi167:Power on completed 2023-12-06T03:45:42.852 INFO:teuthology.provision.fog.smithi167:Waiting for deploy to finish 2023-12-06T03:48:24.854 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:48:27.910 WARNING:teuthology.provision.fog.smithi077:[Errno None] Unable to connect to port 22 on 172.21.15.77 2023-12-06T03:48:28.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:48:31.906 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-12-06T03:48:33.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:48:36.998 WARNING:teuthology.provision.fog.smithi077:[Errno None] Unable to connect to port 22 on 172.21.15.77 2023-12-06T03:48:37.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:48:40.962 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-12-06T03:48:43.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:48:46.086 WARNING:teuthology.provision.fog.smithi077:[Errno None] Unable to connect to port 22 on 172.21.15.77 2023-12-06T03:48:46.965 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:48:50.022 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-12-06T03:48:52.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:48:55.170 WARNING:teuthology.provision.fog.smithi077:[Errno None] Unable to connect to port 22 on 172.21.15.77 2023-12-06T03:48:56.024 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:48:59.074 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-12-06T03:49:01.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:49:04.226 WARNING:teuthology.provision.fog.smithi077:[Errno None] Unable to connect to port 22 on 172.21.15.77 2023-12-06T03:49:05.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:49:08.134 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-12-06T03:49:10.229 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:49:13.282 WARNING:teuthology.provision.fog.smithi077:[Errno None] Unable to connect to port 22 on 172.21.15.77 2023-12-06T03:49:14.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:49:17.186 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-12-06T03:49:19.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:49:23.190 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:49:26.242 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-12-06T03:49:32.246 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:49:35.298 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2023-12-06T03:49:41.302 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:50:19.325 WARNING:teuthology.provision.fog.smithi077:timed out 2023-12-06T03:50:25.332 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:50:25.580 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-06T03:50:25.580 DEBUG:teuthology.orchestra.run.smithi077:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-06T03:50:25.839 INFO:teuthology.provision.fog.smithi077:Node is ready 2023-12-06T03:50:25.840 DEBUG:teuthology.orchestra.run.smithi077:> hostname 2023-12-06T03:50:25.862 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077.front.sepia.ceph.com 2023-12-06T03:50:25.862 DEBUG:teuthology.orchestra.run.smithi077:> grep smithi077.front.sepia.ceph.com /etc/hosts 2023-12-06T03:50:25.922 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77 smithi077.front.sepia.ceph.com smithi077 2023-12-06T03:50:25.922 DEBUG:teuthology.orchestra.run.smithi077:> sudo hostname smithi077 2023-12-06T03:50:26.013 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i -e 's/smithi077.front.sepia.ceph.com/smithi077/g' /etc/hosts 2023-12-06T03:50:26.043 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i -e 's/172.21.15.77/172.21.15.77/g' /etc/hosts 2023-12-06T03:50:26.113 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i -e 's/smithi077.front.sepia.ceph.com/smithi077/g' /etc/hostname 2023-12-06T03:50:26.182 DEBUG:teuthology.orchestra.run.smithi077:> sudo hostname smithi077 2023-12-06T03:50:26.250 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2023-12-06T03:50:26.310 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream" 2023-12-06T03:50:26.311 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="8" 2023-12-06T03:50:26.311 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos" 2023-12-06T03:50:26.311 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora" 2023-12-06T03:50:26.311 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="8" 2023-12-06T03:50:26.311 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el8" 2023-12-06T03:50:26.311 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 8" 2023-12-06T03:50:26.311 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2023-12-06T03:50:26.311 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-12-06T03:50:26.311 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/" 2023-12-06T03:50:26.311 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-06T03:50:26.311 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-06T03:50:26.312 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-06T03:50:26.312 INFO:teuthology.provision.fog.smithi077:Deploy complete! 2023-12-06T03:50:26.314 DEBUG:teuthology.parallel:result is None 2023-12-06T03:50:41.318 WARNING:teuthology.provision.fog.smithi167:timed out 2023-12-06T03:50:47.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:50:47.549 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-06T03:50:47.549 DEBUG:teuthology.orchestra.run.smithi167:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-06T03:50:47.791 INFO:teuthology.provision.fog.smithi167:Node is ready 2023-12-06T03:50:47.792 DEBUG:teuthology.orchestra.run.smithi167:> hostname 2023-12-06T03:50:47.812 INFO:teuthology.orchestra.run.smithi167.stdout:smithi167.front.sepia.ceph.com 2023-12-06T03:50:47.812 DEBUG:teuthology.orchestra.run.smithi167:> grep smithi167.front.sepia.ceph.com /etc/hosts 2023-12-06T03:50:47.872 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.15.167 smithi167.front.sepia.ceph.com smithi167 2023-12-06T03:50:47.872 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname smithi167 2023-12-06T03:50:47.975 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/smithi167.front.sepia.ceph.com/smithi167/g' /etc/hosts 2023-12-06T03:50:48.005 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/172.21.15.167/172.21.15.167/g' /etc/hosts 2023-12-06T03:50:48.073 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/smithi167.front.sepia.ceph.com/smithi167/g' /etc/hostname 2023-12-06T03:50:48.142 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname smithi167 2023-12-06T03:50:48.212 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2023-12-06T03:50:48.271 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2023-12-06T03:50:48.272 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="8" 2023-12-06T03:50:48.272 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2023-12-06T03:50:48.272 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2023-12-06T03:50:48.272 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="8" 2023-12-06T03:50:48.272 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el8" 2023-12-06T03:50:48.272 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 8" 2023-12-06T03:50:48.272 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2023-12-06T03:50:48.272 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-12-06T03:50:48.272 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2023-12-06T03:50:48.272 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-06T03:50:48.273 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-06T03:50:48.273 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-06T03:50:48.273 INFO:teuthology.provision.fog.smithi167:Deploy complete! 2023-12-06T03:50:48.273 DEBUG:teuthology.parallel:result is None 2023-12-06T03:50:48.431 INFO:teuthology.lock.ops:Checking smithi077.front.sepia.ceph.com 2023-12-06T03:50:48.431 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-06T03:50:48.458 INFO:teuthology.lock.ops:Checking smithi167.front.sepia.ceph.com 2023-12-06T03:50:48.458 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-06T03:50:48.484 INFO:teuthology.lock.ops:Updating [smithi077.front.sepia.ceph.com]: set os type and version on server 2023-12-06T03:50:48.484 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2023-12-06T03:50:48.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:50:48.558 DEBUG:teuthology.orchestra.run.smithi077:> true 2023-12-06T03:50:48.701 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2023-12-06T03:50:48.701 DEBUG:teuthology.orchestra.run.smithi077:> uname -m 2023-12-06T03:50:48.760 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2023-12-06T03:50:48.760 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2023-12-06T03:50:48.819 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream" 2023-12-06T03:50:48.819 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="8" 2023-12-06T03:50:48.819 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos" 2023-12-06T03:50:48.820 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora" 2023-12-06T03:50:48.820 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="8" 2023-12-06T03:50:48.820 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el8" 2023-12-06T03:50:48.820 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 8" 2023-12-06T03:50:48.820 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2023-12-06T03:50:48.820 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-12-06T03:50:48.820 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/" 2023-12-06T03:50:48.820 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-06T03:50:48.820 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-06T03:50:48.821 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-06T03:50:48.821 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2023-12-06T03:50:48.842 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: set os type and version on server 2023-12-06T03:50:48.842 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-12-06T03:50:48.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:50:48.909 DEBUG:teuthology.orchestra.run.smithi167:> true 2023-12-06T03:50:48.977 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2023-12-06T03:50:48.977 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2023-12-06T03:50:49.037 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2023-12-06T03:50:49.037 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2023-12-06T03:50:49.096 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2023-12-06T03:50:49.096 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="8" 2023-12-06T03:50:49.096 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2023-12-06T03:50:49.096 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2023-12-06T03:50:49.096 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="8" 2023-12-06T03:50:49.096 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el8" 2023-12-06T03:50:49.096 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 8" 2023-12-06T03:50:49.096 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2023-12-06T03:50:49.097 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-12-06T03:50:49.097 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2023-12-06T03:50:49.097 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-06T03:50:49.097 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-06T03:50:49.097 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-06T03:50:49.098 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2023-12-06T03:50:49.117 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T03:50:49.225 INFO:teuthology.dispatcher.supervisor:Running job 7478730 2023-12-06T03:50:49.226 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478730 --name yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478730/orig.config.yaml 2023-12-06T03:50:49.232 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478730 2023-12-06T03:50:49.232 INFO:teuthology.dispatcher.supervisor:Job PID: 3785596 2023-12-06T03:50:49.232 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-06T03:52:49.330 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T03:54:49.507 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T03:56:49.682 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T03:58:49.859 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T04:00:50.026 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T04:02:50.199 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T04:04:50.359 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T04:06:50.527 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T04:08:50.695 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T04:10:50.863 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T04:12:51.031 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T04:14:51.195 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T04:16:51.367 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T04:16:51.443 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-06T04:16:51.608 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-06T04:16:51.608 INFO:teuthology.nuke:Checking targets against current locks 2023-12-06T04:16:51.655 DEBUG:teuthology.nuke:shortname: smithi077 2023-12-06T04:16:51.655 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-06T04:16:51.677 DEBUG:teuthology.nuke:shortname: smithi167 2023-12-06T04:16:51.678 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-06T04:16:51.682 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478730', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-06 03:45:24.423841', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCKNEit6ECrBLDws+eqhSqKjuhTVsdQMglMla7R42Fyz9u/7r9JhsbjJ5Hiwfs3RKtwW2RzMLFmmmvqDpvVNhd4='} 2023-12-06T04:16:51.706 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478730', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-06 03:45:24.425244', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM8HfKuNaPTp7R7IU5NRFrz9cMwJqi9IIMDzxmwjQjQTFg7/mvAPqoZBYFqFK00orGcLNgwDI905pUx/17E1AIo='} 2023-12-06T04:16:51.733 INFO:teuthology.orchestra.console.smithi077:Power off 2023-12-06T04:16:51.734 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-06T04:16:51.762 DEBUG:teuthology.orchestra.console.smithi077:power off output: Chassis Power Control: Down/Off 2023-12-06T04:16:51.762 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T04:16:51.784 INFO:teuthology.orchestra.console.smithi167:Power off 2023-12-06T04:16:51.784 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-06T04:16:51.792 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is off 2023-12-06T04:16:51.810 DEBUG:teuthology.orchestra.console.smithi167:power off output: Chassis Power Control: Down/Off 2023-12-06T04:16:51.810 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T04:16:51.838 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off 2023-12-06T04:16:51.894 INFO:teuthology.orchestra.console.smithi077:Power off completed 2023-12-06T04:16:51.940 INFO:teuthology.orchestra.console.smithi167:Power off completed 2023-12-06T04:16:52.051 INFO:teuthology.lock.ops:unlocked: smithi077.front.sepia.ceph.com 2023-12-06T04:16:52.092 INFO:teuthology.lock.ops:unlocked: smithi167.front.sepia.ceph.com