2024-01-24T01:11:35.664 INFO:root:teuthology version: 0.0.1.dev251+gc856e0f 2024-01-24T01:11:35.664 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T01:11:35.811 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f474f591c10>, signals=[15]) 2024-01-24T01:11:35.811 INFO:teuthology.lock.ops:Start node 'smithi007.front.sepia.ceph.com' reimaging 2024-01-24T01:11:35.812 INFO:teuthology.lock.ops:Updating [smithi007.front.sepia.ceph.com]: reset os type and version on server 2024-01-24T01:11:35.812 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-01-24T01:11:35.831 INFO:teuthology.lock.ops:Start node 'smithi159.front.sepia.ceph.com' reimaging 2024-01-24T01:11:35.831 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: reset os type and version on server 2024-01-24T01:11:35.831 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-01-24T01:11:35.955 DEBUG:teuthology.provision.fog.smithi007:Requesting image smithi_centos_8.stream (ID 3) 2024-01-24T01:11:35.980 DEBUG:teuthology.provision.fog.smithi159:Requesting image smithi_centos_8.stream (ID 3) 2024-01-24T01:11:36.014 INFO:teuthology.provision.fog.smithi007:Scheduling deploy of centos 8.stream 2024-01-24T01:11:36.030 INFO:teuthology.provision.fog.smithi159:Scheduling deploy of centos 8.stream 2024-01-24T01:11:36.346 INFO:teuthology.orchestra.console.smithi007:Power off 2024-01-24T01:11:36.347 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-24T01:11:36.360 INFO:teuthology.orchestra.console.smithi159:Power off 2024-01-24T01:11:36.360 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-24T01:11:36.369 DEBUG:teuthology.orchestra.console.smithi007:power off output: Chassis Power Control: Down/Off 2024-01-24T01:11:36.369 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T01:11:36.381 DEBUG:teuthology.orchestra.console.smithi159:power off output: Chassis Power Control: Down/Off 2024-01-24T01:11:36.382 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T01:11:36.392 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-01-24T01:11:36.405 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2024-01-24T01:11:40.396 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T01:11:40.406 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T01:11:40.505 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-01-24T01:11:40.514 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2024-01-24T01:11:44.508 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T01:11:44.515 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T01:11:44.617 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2024-01-24T01:11:44.625 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2024-01-24T01:11:44.719 INFO:teuthology.orchestra.console.smithi007:Power off completed 2024-01-24T01:11:44.726 INFO:teuthology.orchestra.console.smithi159:Power off completed 2024-01-24T01:11:44.820 INFO:teuthology.orchestra.console.smithi007:Power on 2024-01-24T01:11:44.820 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-24T01:11:44.828 INFO:teuthology.orchestra.console.smithi159:Power on 2024-01-24T01:11:44.828 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-24T01:11:44.843 DEBUG:teuthology.orchestra.console.smithi007:power on output: Chassis Power Control: Up/On 2024-01-24T01:11:44.843 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T01:11:44.852 DEBUG:teuthology.orchestra.console.smithi159:power on output: Chassis Power Control: Up/On 2024-01-24T01:11:44.852 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T01:11:44.868 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2024-01-24T01:11:44.872 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2024-01-24T01:11:48.872 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T01:11:48.879 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T01:11:48.981 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2024-01-24T01:11:48.987 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2024-01-24T01:11:52.986 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T01:11:52.993 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T01:11:53.096 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-01-24T01:11:53.103 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2024-01-24T01:11:53.197 INFO:teuthology.orchestra.console.smithi007:Power on completed 2024-01-24T01:11:53.204 INFO:teuthology.orchestra.console.smithi159:Power on completed 2024-01-24T01:11:53.298 INFO:teuthology.provision.fog.smithi007:Waiting for deploy to finish 2024-01-24T01:11:53.305 INFO:teuthology.provision.fog.smithi159:Waiting for deploy to finish 2024-01-24T01:14:24.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:14:27.325 WARNING:teuthology.provision.fog.smithi007:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-01-24T01:14:33.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:14:36.380 WARNING:teuthology.provision.fog.smithi007:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-01-24T01:14:39.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:14:42.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:14:42.396 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-01-24T01:14:45.436 WARNING:teuthology.provision.fog.smithi007:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-01-24T01:14:48.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:14:51.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:14:51.452 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-01-24T01:14:54.492 WARNING:teuthology.provision.fog.smithi007:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-01-24T01:14:57.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:15:00.493 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:15:00.508 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-01-24T01:15:03.548 WARNING:teuthology.provision.fog.smithi007:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-01-24T01:15:06.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:15:09.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:15:09.568 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-01-24T01:15:15.573 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:15:18.652 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-01-24T01:15:24.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:15:27.740 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-01-24T01:15:33.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:15:36.828 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-01-24T01:15:42.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:16:09.584 WARNING:teuthology.provision.fog.smithi007:timed out 2024-01-24T01:16:15.589 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:16:15.811 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-24T01:16:15.812 DEBUG:teuthology.orchestra.run.smithi007:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-24T01:16:16.079 INFO:teuthology.provision.fog.smithi007:Node is ready 2024-01-24T01:16:16.080 DEBUG:teuthology.orchestra.run.smithi007:> hostname 2024-01-24T01:16:16.101 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007.front.sepia.ceph.com 2024-01-24T01:16:16.101 DEBUG:teuthology.orchestra.run.smithi007:> grep smithi007.front.sepia.ceph.com /etc/hosts 2024-01-24T01:16:16.163 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7 smithi007.front.sepia.ceph.com smithi007 2024-01-24T01:16:16.163 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname smithi007 2024-01-24T01:16:16.265 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i -e 's/smithi007.front.sepia.ceph.com/smithi007/g' /etc/hosts 2024-01-24T01:16:16.294 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i -e 's/172.21.15.7/172.21.15.7/g' /etc/hosts 2024-01-24T01:16:16.365 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i -e 's/smithi007.front.sepia.ceph.com/smithi007/g' /etc/hostname 2024-01-24T01:16:16.437 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname smithi007 2024-01-24T01:16:16.509 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-01-24T01:16:16.570 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-01-24T01:16:16.570 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="8" 2024-01-24T01:16:16.570 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-01-24T01:16:16.571 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-01-24T01:16:16.571 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="8" 2024-01-24T01:16:16.571 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el8" 2024-01-24T01:16:16.571 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 8" 2024-01-24T01:16:16.571 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-01-24T01:16:16.571 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-01-24T01:16:16.571 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-01-24T01:16:16.571 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-24T01:16:16.571 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-24T01:16:16.571 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-24T01:16:16.572 INFO:teuthology.provision.fog.smithi007:Deploy complete! 2024-01-24T01:16:16.573 DEBUG:teuthology.parallel:result is None 2024-01-24T01:16:42.838 WARNING:teuthology.provision.fog.smithi159:timed out 2024-01-24T01:16:48.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:16:49.090 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-24T01:16:49.090 DEBUG:teuthology.orchestra.run.smithi159:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-24T01:16:49.350 INFO:teuthology.provision.fog.smithi159:Node is ready 2024-01-24T01:16:49.350 DEBUG:teuthology.orchestra.run.smithi159:> hostname 2024-01-24T01:16:49.369 INFO:teuthology.orchestra.run.smithi159.stdout:smithi159.front.sepia.ceph.com 2024-01-24T01:16:49.369 DEBUG:teuthology.orchestra.run.smithi159:> grep smithi159.front.sepia.ceph.com /etc/hosts 2024-01-24T01:16:49.429 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159 smithi159.front.sepia.ceph.com smithi159 2024-01-24T01:16:49.429 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159 2024-01-24T01:16:49.536 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi159.front.sepia.ceph.com/smithi159/g' /etc/hosts 2024-01-24T01:16:49.565 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/172.21.15.159/172.21.15.159/g' /etc/hosts 2024-01-24T01:16:49.635 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi159.front.sepia.ceph.com/smithi159/g' /etc/hostname 2024-01-24T01:16:49.703 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159 2024-01-24T01:16:49.771 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-01-24T01:16:49.828 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-01-24T01:16:49.829 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="8" 2024-01-24T01:16:49.829 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-01-24T01:16:49.829 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-01-24T01:16:49.829 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="8" 2024-01-24T01:16:49.829 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el8" 2024-01-24T01:16:49.829 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 8" 2024-01-24T01:16:49.829 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-01-24T01:16:49.829 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-01-24T01:16:49.829 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-01-24T01:16:49.829 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-24T01:16:49.829 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-24T01:16:49.829 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-24T01:16:49.830 INFO:teuthology.provision.fog.smithi159:Deploy complete! 2024-01-24T01:16:49.830 DEBUG:teuthology.parallel:result is None 2024-01-24T01:16:49.984 INFO:teuthology.lock.ops:Checking smithi007.front.sepia.ceph.com 2024-01-24T01:16:49.984 INFO:teuthology.lock.ops:New key found. Updating... 2024-01-24T01:16:50.009 INFO:teuthology.lock.ops:Checking smithi159.front.sepia.ceph.com 2024-01-24T01:16:50.010 INFO:teuthology.lock.ops:New key found. Updating... 2024-01-24T01:16:50.027 INFO:teuthology.lock.ops:Updating [smithi007.front.sepia.ceph.com]: set os type and version on server 2024-01-24T01:16:50.027 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-01-24T01:16:50.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:16:50.101 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-01-24T01:16:50.240 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-01-24T01:16:50.240 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-01-24T01:16:50.301 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-01-24T01:16:50.301 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-01-24T01:16:50.362 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-01-24T01:16:50.362 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="8" 2024-01-24T01:16:50.362 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-01-24T01:16:50.362 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-01-24T01:16:50.362 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="8" 2024-01-24T01:16:50.362 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el8" 2024-01-24T01:16:50.362 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 8" 2024-01-24T01:16:50.363 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-01-24T01:16:50.363 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-01-24T01:16:50.363 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-01-24T01:16:50.363 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-24T01:16:50.363 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-24T01:16:50.363 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-24T01:16:50.363 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-01-24T01:16:50.385 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: set os type and version on server 2024-01-24T01:16:50.385 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-01-24T01:16:50.385 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-24T01:16:50.459 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-01-24T01:16:50.527 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-01-24T01:16:50.527 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2024-01-24T01:16:50.585 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2024-01-24T01:16:50.586 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-01-24T01:16:50.644 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-01-24T01:16:50.645 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="8" 2024-01-24T01:16:50.645 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-01-24T01:16:50.645 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-01-24T01:16:50.645 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="8" 2024-01-24T01:16:50.645 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el8" 2024-01-24T01:16:50.645 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 8" 2024-01-24T01:16:50.645 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-01-24T01:16:50.645 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-01-24T01:16:50.645 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-01-24T01:16:50.645 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-24T01:16:50.645 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-24T01:16:50.645 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-24T01:16:50.646 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-01-24T01:16:50.668 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T01:16:50.773 INFO:teuthology.dispatcher.supervisor:Running job 7529735 2024-01-24T01:16:50.774 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-01-23_19:22:22-rados-wip-yuri5-testing-2024-01-11-1300-pacific-distro-default-smithi/7529735 --name yuriw-2024-01-23_19:22:22-rados-wip-yuri5-testing-2024-01-11-1300-pacific-distro-default-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/yuriw-2024-01-23_19:22:22-rados-wip-yuri5-testing-2024-01-11-1300-pacific-distro-default-smithi/7529735/orig.config.yaml 2024-01-24T01:16:50.778 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2024-01-23_19:22:22-rados-wip-yuri5-testing-2024-01-11-1300-pacific-distro-default-smithi/7529735 2024-01-24T01:16:50.778 INFO:teuthology.dispatcher.supervisor:Job PID: 3322903 2024-01-24T01:16:50.778 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-01-24T01:18:50.879 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T01:20:51.056 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T01:22:51.228 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T01:24:51.369 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T01:26:51.508 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T01:28:51.680 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T01:30:51.848 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T01:32:52.018 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T01:34:52.188 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T01:36:52.359 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T01:38:52.524 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T01:40:52.694 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T01:42:52.860 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-24T01:42:52.934 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2024-01-24T01:42:53.052 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2024-01-24T01:42:53.053 INFO:teuthology.nuke:Checking targets against current locks 2024-01-24T01:42:53.106 DEBUG:teuthology.nuke:shortname: smithi007 2024-01-24T01:42:53.106 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-24T01:42:53.126 DEBUG:teuthology.nuke:shortname: smithi159 2024-01-24T01:42:53.127 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-24T01:42:53.130 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-23_19:22:22-rados-wip-yuri5-testing-2024-01-11-1300-pacific-distro-default-smithi/7529735', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-24 01:11:34.631205', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC/Y+jdGpXsPztItc7vOXkfdn7pewKOZ+qEO6lkFmb9HPVLvrz5Kji4qUpmhSSqXXeNNEbGAUggUcV+O/m0iAqY='} 2024-01-24T01:42:53.152 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-23_19:22:22-rados-wip-yuri5-testing-2024-01-11-1300-pacific-distro-default-smithi/7529735', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-24 01:11:34.629647', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM0wH7ibOVGnX6aMDLvVjJeYhx91i4w/j7w78dFAePBLxsGczC1MyF0UZAVCruSsPGH+pOJBv6mbSYr+FwgaD0k='} 2024-01-24T01:42:53.179 INFO:teuthology.orchestra.console.smithi007:Power off 2024-01-24T01:42:53.180 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-24T01:42:53.202 DEBUG:teuthology.orchestra.console.smithi007:power off output: Chassis Power Control: Down/Off 2024-01-24T01:42:53.202 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T01:42:53.214 INFO:teuthology.orchestra.console.smithi159:Power off 2024-01-24T01:42:53.214 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-24T01:42:53.222 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2024-01-24T01:42:53.234 DEBUG:teuthology.orchestra.console.smithi159:power off output: Chassis Power Control: Down/Off 2024-01-24T01:42:53.235 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-24T01:42:53.255 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2024-01-24T01:42:53.323 INFO:teuthology.orchestra.console.smithi007:Power off completed 2024-01-24T01:42:53.357 INFO:teuthology.orchestra.console.smithi159:Power off completed 2024-01-24T01:42:53.475 INFO:teuthology.lock.ops:unlocked: smithi007.front.sepia.ceph.com 2024-01-24T01:42:53.499 INFO:teuthology.lock.ops:unlocked: smithi159.front.sepia.ceph.com