2023-11-30T22:29:55.414 INFO:root:teuthology version: 0.0.1.dev212+g595fad1 2023-11-30T22:29:55.417 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T22:29:55.610 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f30efb39550>, signals=[15]) 2023-11-30T22:29:55.611 INFO:teuthology.lock.ops:Start node 'smithi069.front.sepia.ceph.com' reimaging 2023-11-30T22:29:55.611 INFO:teuthology.lock.ops:Updating [smithi069.front.sepia.ceph.com]: reset os type and version on server 2023-11-30T22:29:55.611 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2023-11-30T22:29:55.631 INFO:teuthology.lock.ops:Start node 'smithi111.front.sepia.ceph.com' reimaging 2023-11-30T22:29:55.631 INFO:teuthology.lock.ops:Updating [smithi111.front.sepia.ceph.com]: reset os type and version on server 2023-11-30T22:29:55.631 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2023-11-30T22:29:55.761 DEBUG:teuthology.provision.fog.smithi111:Requesting image smithi_centos_8.stream (ID 3) 2023-11-30T22:29:55.765 DEBUG:teuthology.provision.fog.smithi069:Requesting image smithi_centos_8.stream (ID 3) 2023-11-30T22:29:55.805 INFO:teuthology.provision.fog.smithi111:Scheduling deploy of centos 8.stream 2023-11-30T22:29:55.811 INFO:teuthology.provision.fog.smithi069:Scheduling deploy of centos 8.stream 2023-11-30T22:29:56.137 INFO:teuthology.orchestra.console.smithi111:Power off 2023-11-30T22:29:56.138 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-30T22:29:56.186 DEBUG:teuthology.orchestra.console.smithi111:power off output: Chassis Power Control: Down/Off 2023-11-30T22:29:56.186 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-30T22:29:56.218 INFO:teuthology.orchestra.console.smithi069:Power off 2023-11-30T22:29:56.218 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-30T22:29:56.227 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is off 2023-11-30T22:29:56.246 DEBUG:teuthology.orchestra.console.smithi069:power off output: Chassis Power Control: Down/Off 2023-11-30T22:29:56.247 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-30T22:29:56.279 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2023-11-30T22:29:56.328 INFO:teuthology.orchestra.console.smithi111:Power off completed 2023-11-30T22:29:56.429 INFO:teuthology.orchestra.console.smithi111:Power on 2023-11-30T22:29:56.429 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-30T22:29:56.450 DEBUG:teuthology.orchestra.console.smithi111:power on output: Chassis Power Control: Up/On 2023-11-30T22:29:56.451 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-30T22:29:56.483 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is off 2023-11-30T22:30:00.283 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-30T22:30:00.405 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2023-11-30T22:30:00.484 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-30T22:30:00.613 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on 2023-11-30T22:30:00.714 INFO:teuthology.orchestra.console.smithi111:Power on completed 2023-11-30T22:30:00.815 INFO:teuthology.provision.fog.smithi111:Waiting for deploy to finish 2023-11-30T22:30:04.410 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-30T22:30:04.520 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2023-11-30T22:30:08.522 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-30T22:30:08.636 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off 2023-11-30T22:30:08.740 INFO:teuthology.orchestra.console.smithi069:Power off completed 2023-11-30T22:30:08.840 INFO:teuthology.orchestra.console.smithi069:Power on 2023-11-30T22:30:08.840 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-30T22:30:08.864 DEBUG:teuthology.orchestra.console.smithi069:power on output: Chassis Power Control: Up/On 2023-11-30T22:30:08.865 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-30T22:30:08.887 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off 2023-11-30T22:30:12.890 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-30T22:30:12.999 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off 2023-11-30T22:30:17.002 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-30T22:30:17.183 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2023-11-30T22:30:17.284 INFO:teuthology.orchestra.console.smithi069:Power on completed 2023-11-30T22:30:17.384 INFO:teuthology.provision.fog.smithi069:Waiting for deploy to finish 2023-11-30T22:33:01.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:33:03.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:33:04.866 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-11-30T22:33:06.246 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69 2023-11-30T22:33:10.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:33:12.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:33:13.922 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-11-30T22:33:15.330 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69 2023-11-30T22:33:19.927 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:33:21.332 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:33:22.978 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-11-30T22:33:24.386 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69 2023-11-30T22:33:28.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:33:30.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:33:32.034 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-11-30T22:33:33.446 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69 2023-11-30T22:33:38.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:33:39.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:33:41.091 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-11-30T22:33:42.534 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69 2023-11-30T22:33:47.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:33:48.537 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:33:50.178 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-11-30T22:33:51.618 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69 2023-11-30T22:33:56.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:33:57.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:33:59.234 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-11-30T22:34:05.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:34:08.290 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-11-30T22:34:14.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:34:45.997 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-30T22:34:45.997 DEBUG:teuthology.orchestra.run.smithi111:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-30T22:34:57.623 WARNING:teuthology.provision.fog.smithi069:timed out 2023-11-30T22:35:03.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:35:03.809 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-30T22:35:03.809 DEBUG:teuthology.orchestra.run.smithi069:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-30T22:35:04.074 INFO:teuthology.provision.fog.smithi069:Node is ready 2023-11-30T22:35:04.075 DEBUG:teuthology.orchestra.run.smithi069:> hostname 2023-11-30T22:35:04.095 INFO:teuthology.orchestra.run.smithi069.stdout:smithi069.front.sepia.ceph.com 2023-11-30T22:35:04.096 DEBUG:teuthology.orchestra.run.smithi069:> grep smithi069.front.sepia.ceph.com /etc/hosts 2023-11-30T22:35:04.156 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69 smithi069.front.sepia.ceph.com smithi069 2023-11-30T22:35:04.156 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname smithi069 2023-11-30T22:35:04.266 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i -e 's/smithi069.front.sepia.ceph.com/smithi069/g' /etc/hosts 2023-11-30T22:35:04.297 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i -e 's/172.21.15.69/172.21.15.69/g' /etc/hosts 2023-11-30T22:35:04.368 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i -e 's/smithi069.front.sepia.ceph.com/smithi069/g' /etc/hostname 2023-11-30T22:35:04.437 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname smithi069 2023-11-30T22:35:04.504 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2023-11-30T22:35:04.562 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Stream" 2023-11-30T22:35:04.562 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="8" 2023-11-30T22:35:04.562 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos" 2023-11-30T22:35:04.562 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora" 2023-11-30T22:35:04.562 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="8" 2023-11-30T22:35:04.563 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el8" 2023-11-30T22:35:04.563 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-30T22:35:04.563 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2023-11-30T22:35:04.563 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-30T22:35:04.563 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://centos.org/" 2023-11-30T22:35:04.563 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-30T22:35:04.563 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-30T22:35:04.563 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-30T22:35:04.564 INFO:teuthology.provision.fog.smithi069:Deploy complete! 2023-11-30T22:35:04.565 DEBUG:teuthology.parallel:result is None 2023-11-30T22:35:11.585 INFO:teuthology.provision.fog.smithi111:Node is ready 2023-11-30T22:35:11.585 DEBUG:teuthology.orchestra.run.smithi111:> hostname 2023-11-30T22:35:11.610 INFO:teuthology.orchestra.run.smithi111.stdout:smithi111.front.sepia.ceph.com 2023-11-30T22:35:11.610 DEBUG:teuthology.orchestra.run.smithi111:> grep smithi111.front.sepia.ceph.com /etc/hosts 2023-11-30T22:35:11.671 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111 smithi111.front.sepia.ceph.com smithi111 2023-11-30T22:35:11.671 DEBUG:teuthology.orchestra.run.smithi111:> sudo hostname smithi111 2023-11-30T22:35:11.794 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i -e 's/smithi111.front.sepia.ceph.com/smithi111/g' /etc/hosts 2023-11-30T22:35:11.825 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i -e 's/172.21.15.111/172.21.15.111/g' /etc/hosts 2023-11-30T22:35:11.894 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i -e 's/smithi111.front.sepia.ceph.com/smithi111/g' /etc/hostname 2023-11-30T22:35:11.964 DEBUG:teuthology.orchestra.run.smithi111:> sudo hostname smithi111 2023-11-30T22:35:12.032 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2023-11-30T22:35:12.091 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream" 2023-11-30T22:35:12.091 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="8" 2023-11-30T22:35:12.091 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos" 2023-11-30T22:35:12.091 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora" 2023-11-30T22:35:12.091 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="8" 2023-11-30T22:35:12.091 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el8" 2023-11-30T22:35:12.091 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-30T22:35:12.091 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2023-11-30T22:35:12.091 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-30T22:35:12.091 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/" 2023-11-30T22:35:12.092 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-30T22:35:12.092 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-30T22:35:12.092 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-30T22:35:12.092 INFO:teuthology.provision.fog.smithi111:Deploy complete! 2023-11-30T22:35:12.092 DEBUG:teuthology.parallel:result is None 2023-11-30T22:35:12.240 INFO:teuthology.lock.ops:Checking smithi069.front.sepia.ceph.com 2023-11-30T22:35:12.241 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-30T22:35:12.269 INFO:teuthology.lock.ops:Checking smithi111.front.sepia.ceph.com 2023-11-30T22:35:12.269 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-30T22:35:12.287 INFO:teuthology.lock.ops:Updating [smithi069.front.sepia.ceph.com]: set os type and version on server 2023-11-30T22:35:12.287 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-11-30T22:35:12.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:35:12.356 DEBUG:teuthology.orchestra.run.smithi069:> true 2023-11-30T22:35:12.427 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2023-11-30T22:35:12.427 DEBUG:teuthology.orchestra.run.smithi069:> uname -m 2023-11-30T22:35:12.488 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2023-11-30T22:35:12.488 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2023-11-30T22:35:12.548 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Stream" 2023-11-30T22:35:12.548 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="8" 2023-11-30T22:35:12.548 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos" 2023-11-30T22:35:12.548 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora" 2023-11-30T22:35:12.548 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="8" 2023-11-30T22:35:12.548 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el8" 2023-11-30T22:35:12.549 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-30T22:35:12.549 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2023-11-30T22:35:12.549 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-30T22:35:12.549 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://centos.org/" 2023-11-30T22:35:12.549 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-30T22:35:12.549 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-30T22:35:12.549 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-30T22:35:12.550 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2023-11-30T22:35:12.570 INFO:teuthology.lock.ops:Updating [smithi111.front.sepia.ceph.com]: set os type and version on server 2023-11-30T22:35:12.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-11-30T22:35:12.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-30T22:35:12.634 DEBUG:teuthology.orchestra.run.smithi111:> true 2023-11-30T22:35:12.705 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-11-30T22:35:12.705 DEBUG:teuthology.orchestra.run.smithi111:> uname -m 2023-11-30T22:35:12.764 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2023-11-30T22:35:12.764 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2023-11-30T22:35:12.826 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream" 2023-11-30T22:35:12.826 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="8" 2023-11-30T22:35:12.826 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos" 2023-11-30T22:35:12.826 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora" 2023-11-30T22:35:12.826 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="8" 2023-11-30T22:35:12.826 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el8" 2023-11-30T22:35:12.826 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-30T22:35:12.826 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2023-11-30T22:35:12.826 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-30T22:35:12.826 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/" 2023-11-30T22:35:12.827 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-30T22:35:12.827 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-30T22:35:12.827 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-30T22:35:12.827 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2023-11-30T22:35:12.853 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T22:35:12.954 INFO:teuthology.dispatcher.supervisor:Running job 7472696 2023-11-30T22:35:12.954 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_595fad1080bd379e3c0231a8e02595924c59a9da/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-11-30_16:00:27-rados-wip-yuri8-testing-2023-11-29-0706-pacific-distro-default-smithi/7472696 --name yuriw-2023-11-30_16:00:27-rados-wip-yuri8-testing-2023-11-29-0706-pacific-distro-default-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/yuriw-2023-11-30_16:00:27-rados-wip-yuri8-testing-2023-11-29-0706-pacific-distro-default-smithi/7472696/orig.config.yaml 2023-11-30T22:35:12.957 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-11-30_16:00:27-rados-wip-yuri8-testing-2023-11-29-0706-pacific-distro-default-smithi/7472696 2023-11-30T22:35:12.957 INFO:teuthology.dispatcher.supervisor:Job PID: 3004001 2023-11-30T22:35:12.957 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-30T22:37:13.054 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T22:39:13.223 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T22:41:13.391 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T22:43:13.563 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T22:45:13.731 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T22:47:16.103 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T22:49:16.275 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T22:51:16.440 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T22:53:16.595 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T22:55:16.755 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T22:57:16.931 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T22:59:17.098 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T23:01:17.267 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T23:03:17.447 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T23:05:17.635 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T23:07:17.807 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T23:09:17.975 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T23:11:18.143 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T23:13:18.311 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T23:15:18.483 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-30T23:15:18.551 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-11-30T23:15:18.731 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-30T23:15:18.731 INFO:teuthology.nuke:Checking targets against current locks 2023-11-30T23:15:18.785 DEBUG:teuthology.nuke:shortname: smithi069 2023-11-30T23:15:18.785 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-30T23:15:18.810 DEBUG:teuthology.nuke:shortname: smithi111 2023-11-30T23:15:18.810 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-30T23:15:18.814 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-30_16:00:27-rados-wip-yuri8-testing-2023-11-29-0706-pacific-distro-default-smithi/7472696', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-30 22:29:54.507847', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGdq17+O60JFA6A21RXreKF+eNcvyv6geOZ1Avre0FDo/ih5cAQoQ7WZGWXUftkQHc2Tr6jEv9QAemCXxeUO9MM='} 2023-11-30T23:15:18.841 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-30_16:00:27-rados-wip-yuri8-testing-2023-11-29-0706-pacific-distro-default-smithi/7472696', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-30 22:29:54.509869', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHC9CCMvkJbF35RhevLCuxvfrCYhwA/5b8ZI8pl2kl0TX3nPdx3V2FwyZ7Av3nZpEYiwbV+7V3NY1RypY8olTSg='} 2023-11-30T23:15:18.873 INFO:teuthology.orchestra.console.smithi069:Power off 2023-11-30T23:15:18.873 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-30T23:15:18.896 DEBUG:teuthology.orchestra.console.smithi069:power off output: Chassis Power Control: Down/Off 2023-11-30T23:15:18.896 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-30T23:15:18.912 INFO:teuthology.orchestra.console.smithi111:Power off 2023-11-30T23:15:18.912 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-30T23:15:18.922 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off 2023-11-30T23:15:18.937 DEBUG:teuthology.orchestra.console.smithi111:power off output: Chassis Power Control: Down/Off 2023-11-30T23:15:18.937 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-30T23:15:18.960 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is off 2023-11-30T23:15:19.023 INFO:teuthology.orchestra.console.smithi069:Power off completed 2023-11-30T23:15:19.061 INFO:teuthology.orchestra.console.smithi111:Power off completed 2023-11-30T23:15:19.179 INFO:teuthology.lock.ops:unlocked: smithi069.front.sepia.ceph.com 2023-11-30T23:15:19.209 INFO:teuthology.lock.ops:unlocked: smithi111.front.sepia.ceph.com