2023-12-22T09:25:35.704 INFO:root:teuthology version: 0.0.1.dev226+g9680d2a 2023-12-22T09:25:35.704 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T09:25:35.850 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8222d55940>, signals=[15]) 2023-12-22T09:25:35.851 INFO:teuthology.lock.ops:Start node 'smithi032.front.sepia.ceph.com' reimaging 2023-12-22T09:25:35.851 INFO:teuthology.lock.ops:Updating [smithi032.front.sepia.ceph.com]: reset os type and version on server 2023-12-22T09:25:35.851 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2023-12-22T09:25:35.874 INFO:teuthology.lock.ops:Start node 'smithi055.front.sepia.ceph.com' reimaging 2023-12-22T09:25:35.874 INFO:teuthology.lock.ops:Updating [smithi055.front.sepia.ceph.com]: reset os type and version on server 2023-12-22T09:25:35.874 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2023-12-22T09:25:35.995 DEBUG:teuthology.provision.fog.smithi032:Requesting image smithi_rhel_8.4 (ID 7) 2023-12-22T09:25:36.018 DEBUG:teuthology.provision.fog.smithi055:Requesting image smithi_rhel_8.4 (ID 7) 2023-12-22T09:25:36.043 INFO:teuthology.provision.fog.smithi032:Scheduling deploy of rhel 8.4 2023-12-22T09:25:36.055 INFO:teuthology.provision.fog.smithi055:Scheduling deploy of rhel 8.4 2023-12-22T09:25:36.306 INFO:teuthology.orchestra.console.smithi032:Power off 2023-12-22T09:25:36.307 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-22T09:25:36.323 INFO:teuthology.orchestra.console.smithi055:Power off 2023-12-22T09:25:36.324 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-22T09:25:36.334 DEBUG:teuthology.orchestra.console.smithi032:power off output: Chassis Power Control: Down/Off 2023-12-22T09:25:36.334 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T09:25:36.345 DEBUG:teuthology.orchestra.console.smithi055:power off output: Chassis Power Control: Down/Off 2023-12-22T09:25:36.346 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T09:25:36.358 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2023-12-22T09:25:36.368 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off 2023-12-22T09:25:36.458 INFO:teuthology.orchestra.console.smithi032:Power off completed 2023-12-22T09:25:36.469 INFO:teuthology.orchestra.console.smithi055:Power off completed 2023-12-22T09:25:36.559 INFO:teuthology.orchestra.console.smithi032:Power on 2023-12-22T09:25:36.560 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-22T09:25:36.569 INFO:teuthology.orchestra.console.smithi055:Power on 2023-12-22T09:25:36.570 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-22T09:25:36.582 DEBUG:teuthology.orchestra.console.smithi032:power on output: Chassis Power Control: Up/On 2023-12-22T09:25:36.582 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T09:25:36.592 DEBUG:teuthology.orchestra.console.smithi055:power on output: Chassis Power Control: Up/On 2023-12-22T09:25:36.592 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T09:25:36.603 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2023-12-22T09:25:36.614 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off 2023-12-22T09:25:40.605 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T09:25:40.616 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T09:25:40.718 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2023-12-22T09:25:40.727 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off 2023-12-22T09:25:40.820 INFO:teuthology.orchestra.console.smithi032:Power on completed 2023-12-22T09:25:40.920 INFO:teuthology.provision.fog.smithi032:Waiting for deploy to finish 2023-12-22T09:25:44.729 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T09:25:44.837 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2023-12-22T09:25:44.938 INFO:teuthology.orchestra.console.smithi055:Power on completed 2023-12-22T09:25:45.039 INFO:teuthology.provision.fog.smithi055:Waiting for deploy to finish 2023-12-22T09:26:41.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:26:42.005 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-12-22T09:26:48.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:26:48.149 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-12-22T09:26:54.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:26:54.289 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-12-22T09:27:00.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:27:00.435 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-12-22T09:27:06.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:27:06.581 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-12-22T09:27:12.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:27:12.721 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-12-22T09:27:18.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:27:18.865 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-12-22T09:27:24.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:27:25.014 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-12-22T09:27:31.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:27:31.157 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-12-22T09:27:37.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:27:55.666 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-12-22T09:28:01.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:04.725 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-12-22T09:28:10.733 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:10.860 ERROR:teuthology.orchestra.connection:Error authenticating with smithi032.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-12-22T09:28:14.989 ERROR:teuthology.orchestra.connection:Error authenticating with smithi032.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-12-22T09:28:15.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:18.933 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55 2023-12-22T09:28:22.074 ERROR:teuthology.orchestra.connection:Error authenticating with smithi032.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-12-22T09:28:24.934 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:27.985 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55 2023-12-22T09:28:32.165 ERROR:teuthology.orchestra.connection:Error authenticating with smithi032.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-12-22T09:28:33.986 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:37.041 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55 2023-12-22T09:28:43.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:45.292 ERROR:teuthology.orchestra.connection:Error authenticating with smithi032.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-12-22T09:28:46.097 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55 2023-12-22T09:28:52.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:55.153 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55 2023-12-22T09:29:01.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:29:01.417 ERROR:teuthology.orchestra.connection:Error authenticating with smithi032.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-12-22T09:29:19.634 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55 2023-12-22T09:29:25.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:29:28.689 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55 2023-12-22T09:29:34.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:29:37.745 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55 2023-12-22T09:29:43.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:29:44.009 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-22T09:29:44.009 DEBUG:teuthology.orchestra.run.smithi055:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-22T09:29:59.286 INFO:teuthology.provision.fog.smithi055:Node is ready 2023-12-22T09:29:59.286 DEBUG:teuthology.orchestra.run.smithi055:> hostname 2023-12-22T09:29:59.303 INFO:teuthology.orchestra.run.smithi055.stdout:smithi055.front.sepia.ceph.com 2023-12-22T09:29:59.303 DEBUG:teuthology.orchestra.run.smithi055:> grep smithi055.front.sepia.ceph.com /etc/hosts 2023-12-22T09:29:59.362 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55 smithi055.front.sepia.ceph.com smithi055 2023-12-22T09:29:59.362 DEBUG:teuthology.orchestra.run.smithi055:> sudo hostname smithi055 2023-12-22T09:29:59.498 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i -e 's/smithi055.front.sepia.ceph.com/smithi055/g' /etc/hosts 2023-12-22T09:29:59.523 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i -e 's/172.21.15.55/172.21.15.55/g' /etc/hosts 2023-12-22T09:29:59.591 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i -e 's/smithi055.front.sepia.ceph.com/smithi055/g' /etc/hostname 2023-12-22T09:29:59.657 DEBUG:teuthology.orchestra.run.smithi055:> sudo hostname smithi055 2023-12-22T09:29:59.721 DEBUG:teuthology.orchestra.run.smithi055:> cat /etc/os-release 2023-12-22T09:29:59.776 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="Red Hat Enterprise Linux" 2023-12-22T09:29:59.776 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="8.4 (Ootpa)" 2023-12-22T09:29:59.776 INFO:teuthology.orchestra.run.smithi055.stdout:ID="rhel" 2023-12-22T09:29:59.776 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="fedora" 2023-12-22T09:29:59.776 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="8.4" 2023-12-22T09:29:59.776 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el8" 2023-12-22T09:29:59.776 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-22T09:29:59.776 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31" 2023-12-22T09:29:59.776 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-22T09:29:59.776 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://www.redhat.com/" 2023-12-22T09:29:59.776 INFO:teuthology.orchestra.run.smithi055.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-22T09:29:59.777 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-22T09:29:59.777 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-22T09:29:59.777 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-22T09:29:59.777 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-22T09:29:59.777 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-22T09:29:59.778 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-22T09:29:59.778 INFO:teuthology.provision.fog.smithi055:Deploy complete! 2023-12-22T09:29:59.780 DEBUG:teuthology.parallel:result is None 2023-12-22T09:30:20.424 WARNING:teuthology.provision.fog.smithi032:timed out 2023-12-22T09:30:26.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:30:36.128 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-22T09:30:36.128 DEBUG:teuthology.orchestra.run.smithi032:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-22T09:30:56.305 INFO:teuthology.provision.fog.smithi032:Node is ready 2023-12-22T09:30:56.306 DEBUG:teuthology.orchestra.run.smithi032:> hostname 2023-12-22T09:30:56.321 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032.front.sepia.ceph.com 2023-12-22T09:30:56.322 DEBUG:teuthology.orchestra.run.smithi032:> grep smithi032.front.sepia.ceph.com /etc/hosts 2023-12-22T09:30:56.376 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32 smithi032.front.sepia.ceph.com smithi032 2023-12-22T09:30:56.377 DEBUG:teuthology.orchestra.run.smithi032:> sudo hostname smithi032 2023-12-22T09:30:56.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i -e 's/smithi032.front.sepia.ceph.com/smithi032/g' /etc/hosts 2023-12-22T09:30:56.510 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i -e 's/172.21.15.32/172.21.15.32/g' /etc/hosts 2023-12-22T09:30:56.576 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i -e 's/smithi032.front.sepia.ceph.com/smithi032/g' /etc/hostname 2023-12-22T09:30:56.640 DEBUG:teuthology.orchestra.run.smithi032:> sudo hostname smithi032 2023-12-22T09:30:56.706 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2023-12-22T09:30:56.760 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Red Hat Enterprise Linux" 2023-12-22T09:30:56.760 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="8.4 (Ootpa)" 2023-12-22T09:30:56.760 INFO:teuthology.orchestra.run.smithi032.stdout:ID="rhel" 2023-12-22T09:30:56.761 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="fedora" 2023-12-22T09:30:56.761 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="8.4" 2023-12-22T09:30:56.761 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el8" 2023-12-22T09:30:56.761 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-22T09:30:56.761 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2023-12-22T09:30:56.761 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-22T09:30:56.761 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.redhat.com/" 2023-12-22T09:30:56.761 INFO:teuthology.orchestra.run.smithi032.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-22T09:30:56.761 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-22T09:30:56.761 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-22T09:30:56.761 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-22T09:30:56.761 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-22T09:30:56.761 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-22T09:30:56.762 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-22T09:30:56.762 INFO:teuthology.provision.fog.smithi032:Deploy complete! 2023-12-22T09:30:56.762 DEBUG:teuthology.parallel:result is None 2023-12-22T09:30:56.927 INFO:teuthology.lock.ops:Checking smithi032.front.sepia.ceph.com 2023-12-22T09:30:56.928 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-22T09:30:56.972 INFO:teuthology.lock.ops:Checking smithi055.front.sepia.ceph.com 2023-12-22T09:30:56.972 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-22T09:30:56.989 INFO:teuthology.lock.ops:Updating [smithi032.front.sepia.ceph.com]: set os type and version on server 2023-12-22T09:30:56.989 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-12-22T09:30:56.990 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:30:57.059 DEBUG:teuthology.orchestra.run.smithi032:> true 2023-12-22T09:30:57.127 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-12-22T09:30:57.127 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2023-12-22T09:30:57.182 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2023-12-22T09:30:57.183 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2023-12-22T09:30:57.236 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Red Hat Enterprise Linux" 2023-12-22T09:30:57.237 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="8.4 (Ootpa)" 2023-12-22T09:30:57.237 INFO:teuthology.orchestra.run.smithi032.stdout:ID="rhel" 2023-12-22T09:30:57.237 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="fedora" 2023-12-22T09:30:57.237 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="8.4" 2023-12-22T09:30:57.237 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el8" 2023-12-22T09:30:57.237 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-22T09:30:57.237 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2023-12-22T09:30:57.237 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-22T09:30:57.237 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.redhat.com/" 2023-12-22T09:30:57.237 INFO:teuthology.orchestra.run.smithi032.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-22T09:30:57.238 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-22T09:30:57.238 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-22T09:30:57.238 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-22T09:30:57.238 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-22T09:30:57.238 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-22T09:30:57.238 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-22T09:30:57.238 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2023-12-22T09:30:57.263 INFO:teuthology.lock.ops:Updating [smithi055.front.sepia.ceph.com]: set os type and version on server 2023-12-22T09:30:57.264 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2023-12-22T09:30:57.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:30:57.335 DEBUG:teuthology.orchestra.run.smithi055:> true 2023-12-22T09:30:57.458 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com' 2023-12-22T09:30:57.458 DEBUG:teuthology.orchestra.run.smithi055:> uname -m 2023-12-22T09:30:57.513 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64 2023-12-22T09:30:57.514 DEBUG:teuthology.orchestra.run.smithi055:> cat /etc/os-release 2023-12-22T09:30:57.569 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="Red Hat Enterprise Linux" 2023-12-22T09:30:57.569 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="8.4 (Ootpa)" 2023-12-22T09:30:57.569 INFO:teuthology.orchestra.run.smithi055.stdout:ID="rhel" 2023-12-22T09:30:57.569 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="fedora" 2023-12-22T09:30:57.569 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="8.4" 2023-12-22T09:30:57.570 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el8" 2023-12-22T09:30:57.570 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-22T09:30:57.570 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31" 2023-12-22T09:30:57.570 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-22T09:30:57.570 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://www.redhat.com/" 2023-12-22T09:30:57.570 INFO:teuthology.orchestra.run.smithi055.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-22T09:30:57.570 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-22T09:30:57.570 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-22T09:30:57.570 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-22T09:30:57.570 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-22T09:30:57.570 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-22T09:30:57.570 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-22T09:30:57.571 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2023-12-22T09:30:57.605 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T09:30:57.698 INFO:teuthology.dispatcher.supervisor:Running job 7499458 2023-12-22T09:30:57.698 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_9680d2aba1e76f64ade295c81599c97907457848/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-12-22_08:14:05-fs-rishabh-mdscap-quincy-testing-default-smithi/7499458 --name rishabh-2023-12-22_08:14:05-fs-rishabh-mdscap-quincy-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} -- /home/teuthworker/archive/rishabh-2023-12-22_08:14:05-fs-rishabh-mdscap-quincy-testing-default-smithi/7499458/orig.config.yaml 2023-12-22T09:30:57.702 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-12-22_08:14:05-fs-rishabh-mdscap-quincy-testing-default-smithi/7499458 2023-12-22T09:30:57.703 INFO:teuthology.dispatcher.supervisor:Job PID: 2287302 2023-12-22T09:30:57.703 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-22T09:32:57.704 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T09:34:57.778 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T09:36:57.836 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T09:38:57.914 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T09:38:57.991 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-22T09:38:58.100 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi032 but it was locked by another job: /home/teuthworker/archive/rishabh-2023-12-22_08:19:05-fs-rishabh-mdscap-pacific-testing-default-smithi/7499492 2023-12-22T09:38:58.100 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-22T09:38:58.100 INFO:teuthology.nuke:Checking targets against current locks 2023-12-22T09:38:58.122 INFO:teuthology.nuke:Not nuking smithi032.front.sepia.ceph.com because description doesn't match: rishabh-2023-12-22_08:14:05-fs-rishabh-mdscap-quincy-testing-default-smithi != /home/teuthworker/archive/rishabh-2023-12-22_08:19:05-fs-rishabh-mdscap-pacific-testing-default-smithi/7499492 2023-12-22T09:38:58.168 DEBUG:teuthology.nuke:shortname: smithi055 2023-12-22T09:38:58.168 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-22T09:38:58.190 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi055.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-22_08:14:05-fs-rishabh-mdscap-quincy-testing-default-smithi/7499458', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-22 09:25:34.807590', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLuuL59QtzGq3P3GwZtVcFLgXsi72HkxM5lrhjHU5Pk2NTgayzwzmrLCu/Y6jVZ7FP4dYFpbHq/S69S8SqVGvH0='} 2023-12-22T09:38:58.240 INFO:teuthology.orchestra.console.smithi055:Power off 2023-12-22T09:38:58.240 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-22T09:38:58.266 DEBUG:teuthology.orchestra.console.smithi055:power off output: Chassis Power Control: Down/Off 2023-12-22T09:38:58.266 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T09:38:58.289 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off 2023-12-22T09:38:58.390 INFO:teuthology.orchestra.console.smithi055:Power off completed 2023-12-22T09:38:58.547 INFO:teuthology.lock.ops:unlocked: smithi055.front.sepia.ceph.com 2023-12-22T09:38:58.549 ERROR:teuthology.nuke:Could not nuke the following targets: targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLFwalqdwWAQHVXvTk/En+0UYWKDQiCeJATwBBy6D4U4gE3cyamkljac0ShAaWx3g83ISpzaxyXQT5nL5TCO02Y=