2023-12-08T07:48:17.245 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-08T07:48:17.245 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T07:48:17.379 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f437f297ee0>, signals=[15]) 2023-12-08T07:48:17.380 INFO:teuthology.lock.ops:Start node 'smithi039.front.sepia.ceph.com' reimaging 2023-12-08T07:48:17.380 INFO:teuthology.lock.ops:Updating [smithi039.front.sepia.ceph.com]: reset os type and version on server 2023-12-08T07:48:17.380 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2023-12-08T07:48:17.406 INFO:teuthology.lock.ops:Start node 'smithi071.front.sepia.ceph.com' reimaging 2023-12-08T07:48:17.406 INFO:teuthology.lock.ops:Updating [smithi071.front.sepia.ceph.com]: reset os type and version on server 2023-12-08T07:48:17.406 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2023-12-08T07:48:17.541 DEBUG:teuthology.provision.fog.smithi039:Requesting image smithi_centos_8.stream (ID 3) 2023-12-08T07:48:17.546 DEBUG:teuthology.provision.fog.smithi071:Requesting image smithi_centos_8.stream (ID 3) 2023-12-08T07:48:17.581 INFO:teuthology.provision.fog.smithi039:Scheduling deploy of centos 8.stream 2023-12-08T07:48:17.593 INFO:teuthology.provision.fog.smithi071:Scheduling deploy of centos 8.stream 2023-12-08T07:48:17.859 INFO:teuthology.orchestra.console.smithi039:Power off 2023-12-08T07:48:17.859 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-08T07:48:17.872 INFO:teuthology.orchestra.console.smithi071:Power off 2023-12-08T07:48:17.873 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-08T07:48:17.883 DEBUG:teuthology.orchestra.console.smithi039:power off output: Chassis Power Control: Down/Off 2023-12-08T07:48:17.883 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T07:48:17.896 DEBUG:teuthology.orchestra.console.smithi071:power off output: Chassis Power Control: Down/Off 2023-12-08T07:48:17.896 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T07:48:17.906 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2023-12-08T07:48:17.918 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2023-12-08T07:48:21.911 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T07:48:21.921 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T07:48:22.020 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2023-12-08T07:48:22.032 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2023-12-08T07:48:26.022 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T07:48:26.033 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T07:48:26.132 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is off 2023-12-08T07:48:26.142 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2023-12-08T07:48:26.233 INFO:teuthology.orchestra.console.smithi039:Power off completed 2023-12-08T07:48:26.334 INFO:teuthology.orchestra.console.smithi039:Power on 2023-12-08T07:48:26.335 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-08T07:48:26.356 DEBUG:teuthology.orchestra.console.smithi039:power on output: Chassis Power Control: Up/On 2023-12-08T07:48:26.357 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T07:48:26.380 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is off 2023-12-08T07:48:30.146 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T07:48:30.255 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off 2023-12-08T07:48:30.356 INFO:teuthology.orchestra.console.smithi071:Power off completed 2023-12-08T07:48:30.381 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T07:48:30.457 INFO:teuthology.orchestra.console.smithi071:Power on 2023-12-08T07:48:30.457 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-08T07:48:30.481 DEBUG:teuthology.orchestra.console.smithi071:power on output: Chassis Power Control: Up/On 2023-12-08T07:48:30.481 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T07:48:30.490 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is off 2023-12-08T07:48:30.503 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off 2023-12-08T07:48:34.495 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T07:48:34.504 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T07:48:34.604 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2023-12-08T07:48:34.612 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2023-12-08T07:48:34.705 INFO:teuthology.orchestra.console.smithi039:Power on completed 2023-12-08T07:48:34.713 INFO:teuthology.orchestra.console.smithi071:Power on completed 2023-12-08T07:48:34.807 INFO:teuthology.provision.fog.smithi039:Waiting for deploy to finish 2023-12-08T07:48:34.813 INFO:teuthology.provision.fog.smithi071:Waiting for deploy to finish 2023-12-08T07:51:20.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T07:51:24.002 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-12-08T07:51:30.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T07:51:33.058 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-12-08T07:51:35.988 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T07:51:36.111 ERROR:teuthology.orchestra.connection:Error authenticating with smithi039.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-12-08T07:51:39.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T07:51:42.114 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-12-08T07:51:48.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T07:51:51.170 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-12-08T07:51:57.172 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T07:52:00.230 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-12-08T07:52:06.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T07:52:09.283 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-12-08T07:52:15.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T07:52:40.114 WARNING:teuthology.provision.fog.smithi039:timed out 2023-12-08T07:52:46.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T07:52:49.189 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2023-12-08T07:52:55.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T07:52:58.242 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2023-12-08T07:53:04.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T07:53:15.287 WARNING:teuthology.provision.fog.smithi071:timed out 2023-12-08T07:53:21.289 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T07:53:21.517 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-08T07:53:21.517 DEBUG:teuthology.orchestra.run.smithi071:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-08T07:53:21.750 INFO:teuthology.provision.fog.smithi071:Node is ready 2023-12-08T07:53:21.750 DEBUG:teuthology.orchestra.run.smithi071:> hostname 2023-12-08T07:53:21.769 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071.front.sepia.ceph.com 2023-12-08T07:53:21.770 DEBUG:teuthology.orchestra.run.smithi071:> grep smithi071.front.sepia.ceph.com /etc/hosts 2023-12-08T07:53:21.829 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71 smithi071.front.sepia.ceph.com smithi071 2023-12-08T07:53:21.830 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname smithi071 2023-12-08T07:53:21.935 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i -e 's/smithi071.front.sepia.ceph.com/smithi071/g' /etc/hosts 2023-12-08T07:53:21.965 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i -e 's/172.21.15.71/172.21.15.71/g' /etc/hosts 2023-12-08T07:53:22.033 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i -e 's/smithi071.front.sepia.ceph.com/smithi071/g' /etc/hostname 2023-12-08T07:53:22.101 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname smithi071 2023-12-08T07:53:22.167 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2023-12-08T07:53:22.225 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2023-12-08T07:53:22.225 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="8" 2023-12-08T07:53:22.225 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2023-12-08T07:53:22.225 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2023-12-08T07:53:22.225 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="8" 2023-12-08T07:53:22.226 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el8" 2023-12-08T07:53:22.226 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 8" 2023-12-08T07:53:22.226 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2023-12-08T07:53:22.226 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-12-08T07:53:22.226 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2023-12-08T07:53:22.226 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-08T07:53:22.226 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-08T07:53:22.226 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-08T07:53:22.227 INFO:teuthology.provision.fog.smithi071:Deploy complete! 2023-12-08T07:53:22.228 DEBUG:teuthology.parallel:result is None 2023-12-08T07:54:04.246 WARNING:teuthology.provision.fog.smithi039:timed out 2023-12-08T07:54:10.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T07:54:10.457 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-08T07:54:10.458 DEBUG:teuthology.orchestra.run.smithi039:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-08T07:54:10.726 INFO:teuthology.provision.fog.smithi039:Node is ready 2023-12-08T07:54:10.726 DEBUG:teuthology.orchestra.run.smithi039:> hostname 2023-12-08T07:54:10.745 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039.front.sepia.ceph.com 2023-12-08T07:54:10.746 DEBUG:teuthology.orchestra.run.smithi039:> grep smithi039.front.sepia.ceph.com /etc/hosts 2023-12-08T07:54:10.806 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39 smithi039.front.sepia.ceph.com smithi039 2023-12-08T07:54:10.806 DEBUG:teuthology.orchestra.run.smithi039:> sudo hostname smithi039 2023-12-08T07:54:10.895 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i -e 's/smithi039.front.sepia.ceph.com/smithi039/g' /etc/hosts 2023-12-08T07:54:10.925 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i -e 's/172.21.15.39/172.21.15.39/g' /etc/hosts 2023-12-08T07:54:10.995 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i -e 's/smithi039.front.sepia.ceph.com/smithi039/g' /etc/hostname 2023-12-08T07:54:11.064 DEBUG:teuthology.orchestra.run.smithi039:> sudo hostname smithi039 2023-12-08T07:54:11.132 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2023-12-08T07:54:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="CentOS Stream" 2023-12-08T07:54:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="8" 2023-12-08T07:54:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:ID="centos" 2023-12-08T07:54:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="rhel fedora" 2023-12-08T07:54:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="8" 2023-12-08T07:54:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el8" 2023-12-08T07:54:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="CentOS Stream 8" 2023-12-08T07:54:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2023-12-08T07:54:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-12-08T07:54:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://centos.org/" 2023-12-08T07:54:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-08T07:54:11.192 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-08T07:54:11.192 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-08T07:54:11.192 INFO:teuthology.provision.fog.smithi039:Deploy complete! 2023-12-08T07:54:11.192 DEBUG:teuthology.parallel:result is None 2023-12-08T07:54:11.344 INFO:teuthology.lock.ops:Checking smithi039.front.sepia.ceph.com 2023-12-08T07:54:11.344 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-08T07:54:11.364 INFO:teuthology.lock.ops:Checking smithi071.front.sepia.ceph.com 2023-12-08T07:54:11.364 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-08T07:54:11.382 INFO:teuthology.lock.ops:Updating [smithi039.front.sepia.ceph.com]: set os type and version on server 2023-12-08T07:54:11.382 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-12-08T07:54:11.383 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T07:54:11.449 DEBUG:teuthology.orchestra.run.smithi039:> true 2023-12-08T07:54:11.520 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-12-08T07:54:11.520 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2023-12-08T07:54:11.578 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2023-12-08T07:54:11.578 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2023-12-08T07:54:11.636 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="CentOS Stream" 2023-12-08T07:54:11.636 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="8" 2023-12-08T07:54:11.637 INFO:teuthology.orchestra.run.smithi039.stdout:ID="centos" 2023-12-08T07:54:11.637 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="rhel fedora" 2023-12-08T07:54:11.637 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="8" 2023-12-08T07:54:11.637 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el8" 2023-12-08T07:54:11.637 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="CentOS Stream 8" 2023-12-08T07:54:11.637 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2023-12-08T07:54:11.637 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-12-08T07:54:11.637 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://centos.org/" 2023-12-08T07:54:11.637 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-08T07:54:11.637 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-08T07:54:11.637 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-08T07:54:11.638 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2023-12-08T07:54:11.665 INFO:teuthology.lock.ops:Updating [smithi071.front.sepia.ceph.com]: set os type and version on server 2023-12-08T07:54:11.665 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2023-12-08T07:54:11.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T07:54:11.731 DEBUG:teuthology.orchestra.run.smithi071:> true 2023-12-08T07:54:11.865 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2023-12-08T07:54:11.865 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2023-12-08T07:54:11.924 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2023-12-08T07:54:11.924 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2023-12-08T07:54:11.982 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2023-12-08T07:54:11.983 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="8" 2023-12-08T07:54:11.983 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2023-12-08T07:54:11.983 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2023-12-08T07:54:11.983 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="8" 2023-12-08T07:54:11.983 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el8" 2023-12-08T07:54:11.983 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 8" 2023-12-08T07:54:11.983 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2023-12-08T07:54:11.983 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-12-08T07:54:11.983 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2023-12-08T07:54:11.983 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-08T07:54:11.983 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-08T07:54:11.984 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-08T07:54:11.984 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2023-12-08T07:54:12.006 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T07:54:12.100 INFO:teuthology.dispatcher.supervisor:Running job 7483760 2023-12-08T07:54:12.101 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-12-08_07:46:48-fs-wip-vshankar-testing-20231206.125818-distro-default-smithi/7483760 --name vshankar-2023-12-08_07:46:48-fs-wip-vshankar-testing-20231206.125818-distro-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/vshankar-2023-12-08_07:46:48-fs-wip-vshankar-testing-20231206.125818-distro-default-smithi/7483760/orig.config.yaml 2023-12-08T07:54:12.104 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2023-12-08_07:46:48-fs-wip-vshankar-testing-20231206.125818-distro-default-smithi/7483760 2023-12-08T07:54:12.104 INFO:teuthology.dispatcher.supervisor:Job PID: 2973019 2023-12-08T07:54:12.105 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-08T07:56:12.106 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T07:58:12.192 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:00:12.262 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:02:12.326 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:04:12.402 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:06:12.475 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:08:12.547 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:10:12.617 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:12:12.679 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:14:12.746 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:16:12.812 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:18:12.884 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:20:12.951 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:22:13.018 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:24:13.084 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:26:13.159 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:28:13.222 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:30:13.298 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:32:13.438 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:34:13.509 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:36:13.574 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:38:13.636 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:40:13.705 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:42:13.771 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:44:13.843 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:46:13.908 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:48:13.980 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:50:14.042 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:52:14.114 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:54:14.174 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:56:14.243 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:58:14.315 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:00:14.384 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:02:14.449 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:04:14.535 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:06:14.612 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:08:14.680 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:10:14.752 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:10:14.815 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-08T09:10:15.028 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-08T09:10:15.028 INFO:teuthology.nuke:Checking targets against current locks 2023-12-08T09:10:15.078 DEBUG:teuthology.nuke:shortname: smithi039 2023-12-08T09:10:15.078 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-08T09:10:15.092 DEBUG:teuthology.nuke:shortname: smithi071 2023-12-08T09:10:15.093 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-08T09:10:15.098 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-08_07:46:48-fs-wip-vshankar-testing-20231206.125818-distro-default-smithi/7483760', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-08 07:48:16.293991', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHRKZgxYr5JzHUC0aSyDnwwGUKTD6sxpasjTMD78Gk30m3QOd6GfwfuNmGaBVL0qnH6nHI2imW+FXlwWffa9H2Q='} 2023-12-08T09:10:15.113 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-08_07:46:48-fs-wip-vshankar-testing-20231206.125818-distro-default-smithi/7483760', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-08 07:48:16.292550', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK51Wxg/U2JFW+pOQy077gD520vQaBduWiagna3FQg04S7ZAGAf7aAdviToTYR0JIG9uykHoyCoGWbnanScG9Pg='} 2023-12-08T09:10:15.146 INFO:teuthology.orchestra.console.smithi039:Power off 2023-12-08T09:10:15.146 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-08T09:10:15.159 INFO:teuthology.orchestra.console.smithi071:Power off 2023-12-08T09:10:15.159 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-08T09:10:15.170 DEBUG:teuthology.orchestra.console.smithi039:power off output: Chassis Power Control: Down/Off 2023-12-08T09:10:15.170 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T09:10:15.183 DEBUG:teuthology.orchestra.console.smithi071:power off output: Chassis Power Control: Down/Off 2023-12-08T09:10:15.183 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T09:10:15.200 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is off 2023-12-08T09:10:15.206 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off 2023-12-08T09:10:15.302 INFO:teuthology.orchestra.console.smithi039:Power off completed 2023-12-08T09:10:15.307 INFO:teuthology.orchestra.console.smithi071:Power off completed 2023-12-08T09:10:15.455 INFO:teuthology.lock.ops:unlocked: smithi071.front.sepia.ceph.com 2023-12-08T09:10:15.463 INFO:teuthology.lock.ops:unlocked: smithi039.front.sepia.ceph.com