2023-12-21T03:48:01.322 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-21T03:48:01.322 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-21T03:48:01.479 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f94f8e5d940>, signals=[15]) 2023-12-21T03:48:01.479 INFO:teuthology.lock.ops:Start node 'smithi107.front.sepia.ceph.com' reimaging 2023-12-21T03:48:01.480 INFO:teuthology.lock.ops:Updating [smithi107.front.sepia.ceph.com]: reset os type and version on server 2023-12-21T03:48:01.480 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2023-12-21T03:48:01.500 INFO:teuthology.lock.ops:Start node 'smithi157.front.sepia.ceph.com' reimaging 2023-12-21T03:48:01.500 INFO:teuthology.lock.ops:Updating [smithi157.front.sepia.ceph.com]: reset os type and version on server 2023-12-21T03:48:01.500 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2023-12-21T03:48:01.630 DEBUG:teuthology.provision.fog.smithi157:Requesting image smithi_rhel_8.4 (ID 7) 2023-12-21T03:48:01.633 DEBUG:teuthology.provision.fog.smithi107:Requesting image smithi_rhel_8.4 (ID 7) 2023-12-21T03:48:01.674 INFO:teuthology.provision.fog.smithi107:Scheduling deploy of rhel 8.4 2023-12-21T03:48:01.686 INFO:teuthology.provision.fog.smithi157:Scheduling deploy of rhel 8.4 2023-12-21T03:48:02.043 INFO:teuthology.orchestra.console.smithi107:Power off 2023-12-21T03:48:02.043 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-21T03:48:02.065 DEBUG:teuthology.orchestra.console.smithi107:power off output: Chassis Power Control: Down/Off 2023-12-21T03:48:02.065 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-21T03:48:02.087 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2023-12-21T03:48:02.122 INFO:teuthology.orchestra.console.smithi157:Power off 2023-12-21T03:48:02.122 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-21T03:48:02.144 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2023-12-21T03:48:02.145 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-21T03:48:02.167 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-12-21T03:48:06.089 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-21T03:48:06.168 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-21T03:48:06.198 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2023-12-21T03:48:06.277 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-12-21T03:48:10.203 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-21T03:48:10.278 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-21T03:48:10.312 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is off 2023-12-21T03:48:10.385 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2023-12-21T03:48:10.414 INFO:teuthology.orchestra.console.smithi107:Power off completed 2023-12-21T03:48:10.486 INFO:teuthology.orchestra.console.smithi157:Power off completed 2023-12-21T03:48:10.515 INFO:teuthology.orchestra.console.smithi107:Power on 2023-12-21T03:48:10.515 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-21T03:48:10.537 DEBUG:teuthology.orchestra.console.smithi107:power on output: Chassis Power Control: Up/On 2023-12-21T03:48:10.537 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-21T03:48:10.566 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is off 2023-12-21T03:48:10.587 INFO:teuthology.orchestra.console.smithi157:Power on 2023-12-21T03:48:10.587 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-21T03:48:10.608 DEBUG:teuthology.orchestra.console.smithi157:power on output: Chassis Power Control: Up/On 2023-12-21T03:48:10.608 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-21T03:48:10.629 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2023-12-21T03:48:14.569 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-21T03:48:14.633 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-21T03:48:14.678 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is off 2023-12-21T03:48:14.742 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2023-12-21T03:48:18.680 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-21T03:48:18.743 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-21T03:48:18.789 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2023-12-21T03:48:18.852 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-12-21T03:48:18.890 INFO:teuthology.orchestra.console.smithi107:Power on completed 2023-12-21T03:48:18.953 INFO:teuthology.orchestra.console.smithi157:Power on completed 2023-12-21T03:48:18.991 INFO:teuthology.provision.fog.smithi107:Waiting for deploy to finish 2023-12-21T03:48:19.054 INFO:teuthology.provision.fog.smithi157:Waiting for deploy to finish 2023-12-21T03:51:05.370 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:51:08.433 WARNING:teuthology.provision.fog.smithi107:[Errno None] Unable to connect to port 22 on 172.21.15.107 2023-12-21T03:51:14.435 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:51:17.493 WARNING:teuthology.provision.fog.smithi107:[Errno None] Unable to connect to port 22 on 172.21.15.107 2023-12-21T03:51:20.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:51:23.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:51:23.541 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-12-21T03:51:26.581 WARNING:teuthology.provision.fog.smithi107:[Errno None] Unable to connect to port 22 on 172.21.15.107 2023-12-21T03:51:29.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:51:32.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:51:32.625 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-12-21T03:51:35.665 WARNING:teuthology.provision.fog.smithi107:[Errno None] Unable to connect to port 22 on 172.21.15.107 2023-12-21T03:51:38.628 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:51:41.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:51:41.685 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-12-21T03:51:44.721 WARNING:teuthology.provision.fog.smithi107:[Errno None] Unable to connect to port 22 on 172.21.15.107 2023-12-21T03:51:47.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:51:50.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:51:50.737 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-12-21T03:51:53.777 WARNING:teuthology.provision.fog.smithi107:[Errno None] Unable to connect to port 22 on 172.21.15.107 2023-12-21T03:51:56.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:51:59.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:51:59.800 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-12-21T03:52:02.837 WARNING:teuthology.provision.fog.smithi107:[Errno None] Unable to connect to port 22 on 172.21.15.107 2023-12-21T03:52:05.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:52:08.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:52:08.885 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-12-21T03:52:14.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:52:17.973 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-12-21T03:52:23.981 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:52:27.061 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-12-21T03:52:33.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:52:51.474 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-12-21T03:52:57.481 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:53:00.565 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-12-21T03:53:06.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:53:08.845 WARNING:teuthology.provision.fog.smithi107:timed out 2023-12-21T03:53:09.649 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-12-21T03:53:14.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:53:15.144 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-21T03:53:15.145 DEBUG:teuthology.orchestra.run.smithi107:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-21T03:53:15.397 INFO:teuthology.provision.fog.smithi107:Node is ready 2023-12-21T03:53:15.397 DEBUG:teuthology.orchestra.run.smithi107:> hostname 2023-12-21T03:53:15.456 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107.front.sepia.ceph.com 2023-12-21T03:53:15.456 DEBUG:teuthology.orchestra.run.smithi107:> grep smithi107.front.sepia.ceph.com /etc/hosts 2023-12-21T03:53:15.514 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107 smithi107.front.sepia.ceph.com smithi107 2023-12-21T03:53:15.515 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname smithi107 2023-12-21T03:53:15.625 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i -e 's/smithi107.front.sepia.ceph.com/smithi107/g' /etc/hosts 2023-12-21T03:53:15.650 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i -e 's/172.21.15.107/172.21.15.107/g' /etc/hosts 2023-12-21T03:53:15.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:53:15.718 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i -e 's/smithi107.front.sepia.ceph.com/smithi107/g' /etc/hostname 2023-12-21T03:53:15.786 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname smithi107 2023-12-21T03:53:15.852 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2023-12-21T03:53:15.909 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="Red Hat Enterprise Linux" 2023-12-21T03:53:15.909 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="8.4 (Ootpa)" 2023-12-21T03:53:15.909 INFO:teuthology.orchestra.run.smithi107.stdout:ID="rhel" 2023-12-21T03:53:15.910 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="fedora" 2023-12-21T03:53:15.910 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="8.4" 2023-12-21T03:53:15.910 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el8" 2023-12-21T03:53:15.910 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-21T03:53:15.910 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2023-12-21T03:53:15.910 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-21T03:53:15.910 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://www.redhat.com/" 2023-12-21T03:53:15.910 INFO:teuthology.orchestra.run.smithi107.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-21T03:53:15.910 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-21T03:53:15.910 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-12-21T03:53:15.910 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-21T03:53:15.910 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-21T03:53:15.910 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-21T03:53:15.910 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-21T03:53:15.911 INFO:teuthology.provision.fog.smithi107:Deploy complete! 2023-12-21T03:53:15.913 DEBUG:teuthology.parallel:result is None 2023-12-21T03:53:15.936 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-21T03:53:15.937 DEBUG:teuthology.orchestra.run.smithi157:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-21T03:53:36.243 INFO:teuthology.provision.fog.smithi157:Node is ready 2023-12-21T03:53:36.243 DEBUG:teuthology.orchestra.run.smithi157:> hostname 2023-12-21T03:53:36.259 INFO:teuthology.orchestra.run.smithi157.stdout:smithi157.front.sepia.ceph.com 2023-12-21T03:53:36.259 DEBUG:teuthology.orchestra.run.smithi157:> grep smithi157.front.sepia.ceph.com /etc/hosts 2023-12-21T03:53:36.313 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.15.157 smithi157.front.sepia.ceph.com smithi157 2023-12-21T03:53:36.313 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname smithi157 2023-12-21T03:53:36.459 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i -e 's/smithi157.front.sepia.ceph.com/smithi157/g' /etc/hosts 2023-12-21T03:53:36.483 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i -e 's/172.21.15.157/172.21.15.157/g' /etc/hosts 2023-12-21T03:53:36.545 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i -e 's/smithi157.front.sepia.ceph.com/smithi157/g' /etc/hostname 2023-12-21T03:53:36.606 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname smithi157 2023-12-21T03:53:36.667 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2023-12-21T03:53:36.719 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Red Hat Enterprise Linux" 2023-12-21T03:53:36.719 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8.4 (Ootpa)" 2023-12-21T03:53:36.720 INFO:teuthology.orchestra.run.smithi157.stdout:ID="rhel" 2023-12-21T03:53:36.720 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="fedora" 2023-12-21T03:53:36.720 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8.4" 2023-12-21T03:53:36.720 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8" 2023-12-21T03:53:36.720 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-21T03:53:36.720 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2023-12-21T03:53:36.720 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-21T03:53:36.720 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.redhat.com/" 2023-12-21T03:53:36.720 INFO:teuthology.orchestra.run.smithi157.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-21T03:53:36.720 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-21T03:53:36.720 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-12-21T03:53:36.720 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-21T03:53:36.720 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-21T03:53:36.720 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-21T03:53:36.721 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-21T03:53:36.721 INFO:teuthology.provision.fog.smithi157:Deploy complete! 2023-12-21T03:53:36.725 DEBUG:teuthology.parallel:result is None 2023-12-21T03:53:36.875 INFO:teuthology.lock.ops:Checking smithi107.front.sepia.ceph.com 2023-12-21T03:53:36.876 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-21T03:53:36.896 INFO:teuthology.lock.ops:Checking smithi157.front.sepia.ceph.com 2023-12-21T03:53:36.896 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-21T03:53:36.913 INFO:teuthology.lock.ops:Updating [smithi107.front.sepia.ceph.com]: set os type and version on server 2023-12-21T03:53:36.913 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2023-12-21T03:53:36.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:53:36.983 DEBUG:teuthology.orchestra.run.smithi107:> true 2023-12-21T03:53:37.101 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2023-12-21T03:53:37.101 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2023-12-21T03:53:37.156 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2023-12-21T03:53:37.157 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2023-12-21T03:53:37.212 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="Red Hat Enterprise Linux" 2023-12-21T03:53:37.212 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="8.4 (Ootpa)" 2023-12-21T03:53:37.212 INFO:teuthology.orchestra.run.smithi107.stdout:ID="rhel" 2023-12-21T03:53:37.212 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="fedora" 2023-12-21T03:53:37.213 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="8.4" 2023-12-21T03:53:37.213 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el8" 2023-12-21T03:53:37.213 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-21T03:53:37.213 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2023-12-21T03:53:37.213 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-21T03:53:37.213 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://www.redhat.com/" 2023-12-21T03:53:37.213 INFO:teuthology.orchestra.run.smithi107.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-21T03:53:37.213 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-21T03:53:37.213 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-12-21T03:53:37.213 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-21T03:53:37.213 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-21T03:53:37.213 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-21T03:53:37.214 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-21T03:53:37.214 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2023-12-21T03:53:37.237 INFO:teuthology.lock.ops:Updating [smithi157.front.sepia.ceph.com]: set os type and version on server 2023-12-21T03:53:37.238 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-12-21T03:53:37.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-21T03:53:37.298 DEBUG:teuthology.orchestra.run.smithi157:> true 2023-12-21T03:53:37.364 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-12-21T03:53:37.364 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2023-12-21T03:53:37.419 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2023-12-21T03:53:37.419 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2023-12-21T03:53:37.473 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Red Hat Enterprise Linux" 2023-12-21T03:53:37.473 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8.4 (Ootpa)" 2023-12-21T03:53:37.474 INFO:teuthology.orchestra.run.smithi157.stdout:ID="rhel" 2023-12-21T03:53:37.474 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="fedora" 2023-12-21T03:53:37.474 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8.4" 2023-12-21T03:53:37.474 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8" 2023-12-21T03:53:37.474 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-21T03:53:37.474 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2023-12-21T03:53:37.474 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-21T03:53:37.474 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.redhat.com/" 2023-12-21T03:53:37.474 INFO:teuthology.orchestra.run.smithi157.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-21T03:53:37.474 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-21T03:53:37.474 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-12-21T03:53:37.474 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-21T03:53:37.474 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-21T03:53:37.474 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-21T03:53:37.475 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-21T03:53:37.475 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2023-12-21T03:53:37.496 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-21T03:53:37.581 INFO:teuthology.dispatcher.supervisor:Running job 7499102 2023-12-21T03:53:37.581 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-12-20_19:57:16-rados-wip-yuri7-testing-2023-12-20-0808-pacific-distro-default-smithi/7499102 --name yuriw-2023-12-20_19:57:16-rados-wip-yuri7-testing-2023-12-20-0808-pacific-distro-default-smithi --description rados/cephadm/smoke/{0-nvme-loop distro/rhel_8.4_container_tools_rhel8 fixed-2 mon_election/classic start} -- /home/teuthworker/archive/yuriw-2023-12-20_19:57:16-rados-wip-yuri7-testing-2023-12-20-0808-pacific-distro-default-smithi/7499102/orig.config.yaml 2023-12-21T03:53:37.584 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-12-20_19:57:16-rados-wip-yuri7-testing-2023-12-20-0808-pacific-distro-default-smithi/7499102 2023-12-21T03:53:37.584 INFO:teuthology.dispatcher.supervisor:Job PID: 2405135 2023-12-21T03:53:37.585 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-21T03:55:37.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-21T03:57:37.846 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-21T03:59:38.018 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-21T04:01:38.182 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-21T04:03:38.350 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-21T04:03:38.417 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-21T04:03:38.511 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-21T04:03:38.512 INFO:teuthology.nuke:Checking targets against current locks 2023-12-21T04:03:38.563 DEBUG:teuthology.nuke:shortname: smithi107 2023-12-21T04:03:38.563 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-21T04:03:38.582 DEBUG:teuthology.nuke:shortname: smithi157 2023-12-21T04:03:38.582 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-21T04:03:38.586 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-20_19:57:16-rados-wip-yuri7-testing-2023-12-20-0808-pacific-distro-default-smithi/7499102', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-21 03:48:00.512923', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJRKjGpGpQJgqUfn7nPPmc5OiwbaVMg991+v2on9MrnW5sXnfSUPEY3SXrcnJJZgMgozYuLpOaF+Lb1AARjN+Js='} 2023-12-21T04:03:38.603 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-20_19:57:16-rados-wip-yuri7-testing-2023-12-20-0808-pacific-distro-default-smithi/7499102', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-21 03:48:00.510925', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHx3Pn/2b4tmNaqYjvuvpUuEVY0QHEVkKkjkOshtZ2nXSmrPf50aveOfTzKE+JPtOXUMwrhdO4zJJOzVmA2Zzng='} 2023-12-21T04:03:38.634 INFO:teuthology.orchestra.console.smithi107:Power off 2023-12-21T04:03:38.635 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-21T04:03:38.658 DEBUG:teuthology.orchestra.console.smithi107:power off output: Chassis Power Control: Down/Off 2023-12-21T04:03:38.659 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-21T04:03:38.683 INFO:teuthology.orchestra.console.smithi157:Power off 2023-12-21T04:03:38.684 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-21T04:03:38.697 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is off 2023-12-21T04:03:38.709 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2023-12-21T04:03:38.709 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-21T04:03:38.732 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2023-12-21T04:03:38.798 INFO:teuthology.orchestra.console.smithi107:Power off completed 2023-12-21T04:03:38.833 INFO:teuthology.orchestra.console.smithi157:Power off completed 2023-12-21T04:03:38.959 INFO:teuthology.lock.ops:unlocked: smithi107.front.sepia.ceph.com 2023-12-21T04:03:38.982 INFO:teuthology.lock.ops:unlocked: smithi157.front.sepia.ceph.com