2023-11-20T19:07:26.192 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-20T19:07:26.193 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:07:26.333 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbe6ee39e50>, signals=[15]) 2023-11-20T19:07:26.334 INFO:teuthology.lock.ops:Start node 'smithi116.front.sepia.ceph.com' reimaging 2023-11-20T19:07:26.334 INFO:teuthology.lock.ops:Updating [smithi116.front.sepia.ceph.com]: reset os type and version on server 2023-11-20T19:07:26.334 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2023-11-20T19:07:26.357 INFO:teuthology.lock.ops:Start node 'smithi181.front.sepia.ceph.com' reimaging 2023-11-20T19:07:26.358 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: reset os type and version on server 2023-11-20T19:07:26.358 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-11-20T19:07:26.481 DEBUG:teuthology.provision.fog.smithi116:Requesting image smithi_centos_8.stream (ID 3) 2023-11-20T19:07:26.492 DEBUG:teuthology.provision.fog.smithi181:Requesting image smithi_centos_8.stream (ID 3) 2023-11-20T19:07:26.524 INFO:teuthology.provision.fog.smithi116:Scheduling deploy of centos 8.stream 2023-11-20T19:07:26.540 INFO:teuthology.provision.fog.smithi181:Scheduling deploy of centos 8.stream 2023-11-20T19:07:26.736 INFO:teuthology.orchestra.console.smithi116:Power off 2023-11-20T19:07:26.736 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-20T19:07:26.757 DEBUG:teuthology.orchestra.console.smithi116:power off output: Chassis Power Control: Down/Off 2023-11-20T19:07:26.757 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-20T19:07:26.771 INFO:teuthology.orchestra.console.smithi181:Power off 2023-11-20T19:07:26.772 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-20T19:07:26.779 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is on 2023-11-20T19:07:26.791 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off 2023-11-20T19:07:26.792 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-20T19:07:26.815 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-11-20T19:07:30.784 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-20T19:07:30.815 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-20T19:07:30.892 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is on 2023-11-20T19:07:30.923 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-11-20T19:07:34.897 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-20T19:07:34.925 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-20T19:07:35.007 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is off 2023-11-20T19:07:35.032 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-11-20T19:07:35.108 INFO:teuthology.orchestra.console.smithi116:Power off completed 2023-11-20T19:07:35.133 INFO:teuthology.orchestra.console.smithi181:Power off completed 2023-11-20T19:07:35.209 INFO:teuthology.orchestra.console.smithi116:Power on 2023-11-20T19:07:35.209 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-20T19:07:35.229 DEBUG:teuthology.orchestra.console.smithi116:power on output: Chassis Power Control: Up/On 2023-11-20T19:07:35.229 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-20T19:07:35.235 INFO:teuthology.orchestra.console.smithi181:Power on 2023-11-20T19:07:35.236 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-20T19:07:35.249 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is off 2023-11-20T19:07:35.256 DEBUG:teuthology.orchestra.console.smithi181:power on output: Chassis Power Control: Up/On 2023-11-20T19:07:35.256 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-20T19:07:35.276 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-11-20T19:07:39.254 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-20T19:07:39.278 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-20T19:07:39.363 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is off 2023-11-20T19:07:39.385 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-11-20T19:07:43.368 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-20T19:07:43.386 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-20T19:07:43.478 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is on 2023-11-20T19:07:43.494 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-11-20T19:07:43.579 INFO:teuthology.orchestra.console.smithi116:Power on completed 2023-11-20T19:07:43.595 INFO:teuthology.orchestra.console.smithi181:Power on completed 2023-11-20T19:07:43.679 INFO:teuthology.provision.fog.smithi116:Waiting for deploy to finish 2023-11-20T19:07:43.696 INFO:teuthology.provision.fog.smithi181:Waiting for deploy to finish 2023-11-20T19:10:14.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-20T19:10:14.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-20T19:10:32.803 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-11-20T19:10:32.806 WARNING:teuthology.provision.fog.smithi116:[Errno None] Unable to connect to port 22 on 172.21.15.116 2023-11-20T19:10:38.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-20T19:10:38.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-20T19:10:41.862 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-11-20T19:10:41.890 WARNING:teuthology.provision.fog.smithi116:[Errno None] Unable to connect to port 22 on 172.21.15.116 2023-11-20T19:10:47.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-20T19:10:47.892 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-20T19:10:50.946 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-11-20T19:10:50.946 WARNING:teuthology.provision.fog.smithi116:[Errno None] Unable to connect to port 22 on 172.21.15.116 2023-11-20T19:10:56.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-20T19:10:56.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-20T19:11:00.002 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-11-20T19:11:00.003 WARNING:teuthology.provision.fog.smithi116:[Errno None] Unable to connect to port 22 on 172.21.15.116 2023-11-20T19:11:06.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-20T19:11:06.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-20T19:11:09.062 WARNING:teuthology.provision.fog.smithi116:[Errno None] Unable to connect to port 22 on 172.21.15.116 2023-11-20T19:11:09.062 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181 2023-11-20T19:11:15.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-20T19:11:15.068 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-20T19:12:15.126 WARNING:teuthology.provision.fog.smithi116:timed out 2023-11-20T19:12:15.127 WARNING:teuthology.provision.fog.smithi181:timed out 2023-11-20T19:12:21.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-20T19:12:21.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-20T19:12:21.345 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-20T19:12:21.345 DEBUG:teuthology.orchestra.run.smithi181:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-20T19:12:21.377 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-20T19:12:21.377 DEBUG:teuthology.orchestra.run.smithi116:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-20T19:12:21.597 INFO:teuthology.provision.fog.smithi181:Node is ready 2023-11-20T19:12:21.597 DEBUG:teuthology.orchestra.run.smithi181:> hostname 2023-11-20T19:12:21.616 INFO:teuthology.orchestra.run.smithi181.stdout:smithi181.front.sepia.ceph.com 2023-11-20T19:12:21.616 DEBUG:teuthology.orchestra.run.smithi181:> grep smithi181.front.sepia.ceph.com /etc/hosts 2023-11-20T19:12:21.638 INFO:teuthology.provision.fog.smithi116:Node is ready 2023-11-20T19:12:21.638 DEBUG:teuthology.orchestra.run.smithi116:> hostname 2023-11-20T19:12:21.656 INFO:teuthology.orchestra.run.smithi116.stdout:smithi116.front.sepia.ceph.com 2023-11-20T19:12:21.657 DEBUG:teuthology.orchestra.run.smithi116:> grep smithi116.front.sepia.ceph.com /etc/hosts 2023-11-20T19:12:21.675 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181 smithi181.front.sepia.ceph.com smithi181 2023-11-20T19:12:21.675 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181 2023-11-20T19:12:21.717 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116 smithi116.front.sepia.ceph.com smithi116 2023-11-20T19:12:21.717 DEBUG:teuthology.orchestra.run.smithi116:> sudo hostname smithi116 2023-11-20T19:12:21.782 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hosts 2023-11-20T19:12:21.811 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/172.21.15.181/172.21.15.181/g' /etc/hosts 2023-11-20T19:12:21.817 DEBUG:teuthology.orchestra.run.smithi116:> sudo sed -i -e 's/smithi116.front.sepia.ceph.com/smithi116/g' /etc/hosts 2023-11-20T19:12:21.847 DEBUG:teuthology.orchestra.run.smithi116:> sudo sed -i -e 's/172.21.15.116/172.21.15.116/g' /etc/hosts 2023-11-20T19:12:21.879 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hostname 2023-11-20T19:12:21.916 DEBUG:teuthology.orchestra.run.smithi116:> sudo sed -i -e 's/smithi116.front.sepia.ceph.com/smithi116/g' /etc/hostname 2023-11-20T19:12:21.946 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181 2023-11-20T19:12:21.984 DEBUG:teuthology.orchestra.run.smithi116:> sudo hostname smithi116 2023-11-20T19:12:22.013 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-11-20T19:12:22.051 DEBUG:teuthology.orchestra.run.smithi116:> cat /etc/os-release 2023-11-20T19:12:22.072 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2023-11-20T19:12:22.072 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8" 2023-11-20T19:12:22.072 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2023-11-20T19:12:22.072 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2023-11-20T19:12:22.072 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8" 2023-11-20T19:12:22.072 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2023-11-20T19:12:22.072 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-20T19:12:22.072 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-11-20T19:12:22.072 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-20T19:12:22.072 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2023-11-20T19:12:22.072 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-20T19:12:22.073 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-20T19:12:22.073 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-20T19:12:22.073 INFO:teuthology.provision.fog.smithi181:Deploy complete! 2023-11-20T19:12:22.077 DEBUG:teuthology.parallel:result is None 2023-11-20T19:12:22.109 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="CentOS Stream" 2023-11-20T19:12:22.109 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="8" 2023-11-20T19:12:22.109 INFO:teuthology.orchestra.run.smithi116.stdout:ID="centos" 2023-11-20T19:12:22.110 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE="rhel fedora" 2023-11-20T19:12:22.110 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="8" 2023-11-20T19:12:22.110 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM_ID="platform:el8" 2023-11-20T19:12:22.110 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-20T19:12:22.110 INFO:teuthology.orchestra.run.smithi116.stdout:ANSI_COLOR="0;31" 2023-11-20T19:12:22.110 INFO:teuthology.orchestra.run.smithi116.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-20T19:12:22.110 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://centos.org/" 2023-11-20T19:12:22.110 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-20T19:12:22.110 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-20T19:12:22.110 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-20T19:12:22.110 INFO:teuthology.provision.fog.smithi116:Deploy complete! 2023-11-20T19:12:22.111 DEBUG:teuthology.parallel:result is None 2023-11-20T19:12:22.254 INFO:teuthology.lock.ops:Checking smithi116.front.sepia.ceph.com 2023-11-20T19:12:22.255 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-20T19:12:22.283 INFO:teuthology.lock.ops:Checking smithi181.front.sepia.ceph.com 2023-11-20T19:12:22.283 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-20T19:12:22.309 INFO:teuthology.lock.ops:Updating [smithi116.front.sepia.ceph.com]: set os type and version on server 2023-11-20T19:12:22.309 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2023-11-20T19:12:22.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-20T19:12:22.376 DEBUG:teuthology.orchestra.run.smithi116:> true 2023-11-20T19:12:22.443 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi116.front.sepia.ceph.com' 2023-11-20T19:12:22.443 DEBUG:teuthology.orchestra.run.smithi116:> uname -m 2023-11-20T19:12:22.501 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2023-11-20T19:12:22.502 DEBUG:teuthology.orchestra.run.smithi116:> cat /etc/os-release 2023-11-20T19:12:22.560 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="CentOS Stream" 2023-11-20T19:12:22.560 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="8" 2023-11-20T19:12:22.560 INFO:teuthology.orchestra.run.smithi116.stdout:ID="centos" 2023-11-20T19:12:22.560 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE="rhel fedora" 2023-11-20T19:12:22.560 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="8" 2023-11-20T19:12:22.560 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM_ID="platform:el8" 2023-11-20T19:12:22.560 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-20T19:12:22.560 INFO:teuthology.orchestra.run.smithi116.stdout:ANSI_COLOR="0;31" 2023-11-20T19:12:22.560 INFO:teuthology.orchestra.run.smithi116.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-20T19:12:22.561 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://centos.org/" 2023-11-20T19:12:22.561 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-20T19:12:22.561 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-20T19:12:22.561 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-20T19:12:22.561 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2023-11-20T19:12:22.583 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: set os type and version on server 2023-11-20T19:12:22.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-11-20T19:12:22.584 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-20T19:12:22.648 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-11-20T19:12:22.716 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-11-20T19:12:22.717 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2023-11-20T19:12:22.774 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2023-11-20T19:12:22.775 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-11-20T19:12:22.833 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2023-11-20T19:12:22.833 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8" 2023-11-20T19:12:22.833 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2023-11-20T19:12:22.833 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2023-11-20T19:12:22.833 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8" 2023-11-20T19:12:22.833 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2023-11-20T19:12:22.833 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-20T19:12:22.833 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-11-20T19:12:22.834 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-20T19:12:22.834 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2023-11-20T19:12:22.834 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-20T19:12:22.834 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-20T19:12:22.834 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-20T19:12:22.834 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-11-20T19:12:22.858 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:12:22.952 INFO:teuthology.dispatcher.supervisor:Running job 7463586 2023-11-20T19:12:22.952 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/virtualenv/bin/teuthology -v --owner scheduled_gabrioux@teuthology --archive /home/teuthworker/archive/gabrioux-2023-11-20_15:58:13-orch:cephadm-wip-teo-track_58812-distro-default-smithi/7463586 --name gabrioux-2023-11-20_15:58:13-orch:cephadm-wip-teo-track_58812-distro-default-smithi --description orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/gabrioux-2023-11-20_15:58:13-orch:cephadm-wip-teo-track_58812-distro-default-smithi/7463586/orig.config.yaml 2023-11-20T19:12:22.954 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/gabrioux-2023-11-20_15:58:13-orch:cephadm-wip-teo-track_58812-distro-default-smithi/7463586 2023-11-20T19:12:22.955 INFO:teuthology.dispatcher.supervisor:Job PID: 3591159 2023-11-20T19:12:22.955 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-20T19:14:23.054 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:16:23.215 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:18:23.379 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:20:23.555 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:22:23.724 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:24:23.887 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:26:24.051 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:28:24.220 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:30:24.391 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:32:24.557 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:34:24.727 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:36:24.885 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:38:25.056 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:40:25.173 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:42:25.299 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:44:25.462 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:46:25.634 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:48:25.803 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:50:25.971 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:52:26.127 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:54:26.298 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:56:26.471 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T19:58:26.618 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:00:26.757 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:02:26.930 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:04:27.098 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:06:27.259 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:08:27.406 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:10:27.555 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:12:27.716 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:14:27.937 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:16:28.106 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:18:28.249 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:20:28.348 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:22:28.515 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:24:28.662 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:26:28.832 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:28:28.999 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:30:29.170 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:32:29.291 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:34:29.439 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:36:29.582 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:38:29.751 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:40:29.922 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-20T20:42:30.014 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-21T02:32:09.531 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-21T02:32:09.591 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-11-21T02:32:09.883 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-21T02:32:09.883 INFO:teuthology.nuke:Checking targets against current locks 2023-11-21T02:32:09.929 DEBUG:teuthology.nuke:shortname: smithi116 2023-11-21T02:32:09.930 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-21T02:32:09.954 DEBUG:teuthology.nuke:shortname: smithi181 2023-11-21T02:32:09.954 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-21T02:32:09.955 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi116.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2023-11-20_15:58:13-orch:cephadm-wip-teo-track_58812-distro-default-smithi/7463586', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-20 19:07:25.424169', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP7tXajikXDBKVv+YzF/qBafwiokYQpHHOqzSIcu1/pLP6jwK4NtnjrH2f+hnXCt1cYpYF3uYWyA/lKqOqHUzsU='} 2023-11-21T02:32:09.977 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2023-11-20_15:58:13-orch:cephadm-wip-teo-track_58812-distro-default-smithi/7463586', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-20 19:07:25.426143', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKjceEMX7Ez/3rpYuH0Er5lvY14zH4apgio0toFX49i2g4gEyoVexELXpuLDyUKklO0F4H5VOT20zaaDwBfYkfk='} 2023-11-21T02:32:10.002 INFO:teuthology.orchestra.console.smithi116:Power off 2023-11-21T02:32:10.002 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-21T02:32:10.022 DEBUG:teuthology.orchestra.console.smithi116:power off output: Chassis Power Control: Down/Off 2023-11-21T02:32:10.022 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-21T02:32:10.036 INFO:teuthology.orchestra.console.smithi181:Power off 2023-11-21T02:32:10.036 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-21T02:32:10.046 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is off 2023-11-21T02:32:10.056 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off 2023-11-21T02:32:10.056 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-21T02:32:10.076 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-11-21T02:32:10.147 INFO:teuthology.orchestra.console.smithi116:Power off completed 2023-11-21T02:32:10.177 INFO:teuthology.orchestra.console.smithi181:Power off completed 2023-11-21T02:32:10.308 INFO:teuthology.lock.ops:unlocked: smithi116.front.sepia.ceph.com 2023-11-21T02:32:10.323 INFO:teuthology.lock.ops:unlocked: smithi181.front.sepia.ceph.com