2024-01-27T13:17:53.453 INFO:root:teuthology version: 0.0.1.dev251+gc856e0f 2024-01-27T13:17:53.454 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-27T13:17:53.597 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f025702ec10>, signals=[15]) 2024-01-27T13:17:53.598 INFO:teuthology.lock.ops:Start node 'smithi119.front.sepia.ceph.com' reimaging 2024-01-27T13:17:53.598 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: reset os type and version on server 2024-01-27T13:17:53.598 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2024-01-27T13:17:53.617 INFO:teuthology.lock.ops:Start node 'smithi163.front.sepia.ceph.com' reimaging 2024-01-27T13:17:53.617 INFO:teuthology.lock.ops:Updating [smithi163.front.sepia.ceph.com]: reset os type and version on server 2024-01-27T13:17:53.617 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2024-01-27T13:17:53.747 DEBUG:teuthology.provision.fog.smithi119:Requesting image smithi_rhel_8.6 (ID 5) 2024-01-27T13:17:53.760 DEBUG:teuthology.provision.fog.smithi163:Requesting image smithi_rhel_8.6 (ID 5) 2024-01-27T13:17:53.805 INFO:teuthology.provision.fog.smithi119:Scheduling deploy of rhel 8.6 2024-01-27T13:17:53.812 INFO:teuthology.provision.fog.smithi163:Scheduling deploy of rhel 8.6 2024-01-27T13:17:54.070 INFO:teuthology.orchestra.console.smithi119:Power off 2024-01-27T13:17:54.070 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-27T13:17:54.090 INFO:teuthology.orchestra.console.smithi163:Power off 2024-01-27T13:17:54.090 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-27T13:17:54.100 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off 2024-01-27T13:17:54.101 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T13:17:54.114 DEBUG:teuthology.orchestra.console.smithi163:power off output: Chassis Power Control: Down/Off 2024-01-27T13:17:54.115 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T13:17:54.125 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2024-01-27T13:17:54.139 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-01-27T13:17:58.128 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T13:17:58.140 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T13:17:58.237 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2024-01-27T13:17:58.251 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-01-27T13:18:02.243 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T13:18:02.252 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T13:18:02.354 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off 2024-01-27T13:18:02.364 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-01-27T13:18:02.456 INFO:teuthology.orchestra.console.smithi119:Power off completed 2024-01-27T13:18:02.556 INFO:teuthology.orchestra.console.smithi119:Power on 2024-01-27T13:18:02.557 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-27T13:18:02.578 DEBUG:teuthology.orchestra.console.smithi119:power on output: Chassis Power Control: Up/On 2024-01-27T13:18:02.578 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T13:18:02.599 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off 2024-01-27T13:18:06.369 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T13:18:06.479 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off 2024-01-27T13:18:06.580 INFO:teuthology.orchestra.console.smithi163:Power off completed 2024-01-27T13:18:06.600 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T13:18:06.681 INFO:teuthology.orchestra.console.smithi163:Power on 2024-01-27T13:18:06.682 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-27T13:18:06.703 DEBUG:teuthology.orchestra.console.smithi163:power on output: Chassis Power Control: Up/On 2024-01-27T13:18:06.703 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T13:18:06.713 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off 2024-01-27T13:18:06.725 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off 2024-01-27T13:18:10.714 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T13:18:10.727 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T13:18:10.823 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2024-01-27T13:18:10.835 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-01-27T13:18:10.925 INFO:teuthology.orchestra.console.smithi119:Power on completed 2024-01-27T13:18:10.936 INFO:teuthology.orchestra.console.smithi163:Power on completed 2024-01-27T13:18:11.026 INFO:teuthology.provision.fog.smithi119:Waiting for deploy to finish 2024-01-27T13:18:11.037 INFO:teuthology.provision.fog.smithi163:Waiting for deploy to finish 2024-01-27T13:20:57.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:21:00.092 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-01-27T13:21:06.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:21:09.152 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-01-27T13:21:12.109 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:21:15.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:21:15.168 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-01-27T13:21:18.236 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-01-27T13:21:21.172 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:21:24.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:21:24.252 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-01-27T13:21:27.296 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-01-27T13:21:30.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:21:33.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:21:33.308 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-01-27T13:21:36.380 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-01-27T13:21:39.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:21:42.364 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-01-27T13:21:42.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:21:45.436 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-01-27T13:21:48.366 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:21:51.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:21:51.452 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-01-27T13:21:57.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:22:00.508 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-01-27T13:22:06.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:22:09.596 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-01-27T13:22:09.852 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-01-27T13:22:15.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:22:15.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:22:18.688 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-01-27T13:22:18.912 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-01-27T13:22:24.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:22:24.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:22:27.772 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-01-27T13:22:28.086 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-27T13:22:28.086 DEBUG:teuthology.orchestra.run.smithi119:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-27T13:22:33.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:22:48.334 INFO:teuthology.provision.fog.smithi119:Node is ready 2024-01-27T13:22:48.334 DEBUG:teuthology.orchestra.run.smithi119:> hostname 2024-01-27T13:22:48.355 INFO:teuthology.orchestra.run.smithi119.stdout:smithi045 2024-01-27T13:22:48.356 DEBUG:teuthology.orchestra.run.smithi119:> grep smithi045 /etc/hosts 2024-01-27T13:22:48.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-27T13:22:48.415 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i -e 's/smithi045/smithi119/g' /etc/hostname 2024-01-27T13:22:48.550 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname smithi119 2024-01-27T13:22:48.580 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2024-01-27T13:22:48.638 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Red Hat Enterprise Linux" 2024-01-27T13:22:48.639 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="8.6 (Ootpa)" 2024-01-27T13:22:48.639 INFO:teuthology.orchestra.run.smithi119.stdout:ID="rhel" 2024-01-27T13:22:48.639 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="fedora" 2024-01-27T13:22:48.639 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="8.6" 2024-01-27T13:22:48.639 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el8" 2024-01-27T13:22:48.639 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-27T13:22:48.639 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2024-01-27T13:22:48.639 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-27T13:22:48.639 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.redhat.com/" 2024-01-27T13:22:48.639 INFO:teuthology.orchestra.run.smithi119.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-27T13:22:48.639 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-27T13:22:48.640 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-01-27T13:22:48.640 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-27T13:22:48.640 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-27T13:22:48.640 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-27T13:22:48.640 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-27T13:22:48.641 INFO:teuthology.provision.fog.smithi119:Deploy complete! 2024-01-27T13:22:48.642 DEBUG:teuthology.parallel:result is None 2024-01-27T13:23:33.795 WARNING:teuthology.provision.fog.smithi163:timed out 2024-01-27T13:23:39.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:23:40.072 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-27T13:23:40.072 DEBUG:teuthology.orchestra.run.smithi163:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-27T13:23:40.329 INFO:teuthology.provision.fog.smithi163:Node is ready 2024-01-27T13:23:40.329 DEBUG:teuthology.orchestra.run.smithi163:> hostname 2024-01-27T13:23:40.347 INFO:teuthology.orchestra.run.smithi163.stdout:smithi163 2024-01-27T13:23:40.348 DEBUG:teuthology.orchestra.run.smithi163:> grep smithi163 /etc/hosts 2024-01-27T13:23:40.404 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.15.163 smithi163.front.sepia.ceph.com smithi163 2024-01-27T13:23:40.405 DEBUG:teuthology.orchestra.run.smithi163:> sudo hostname smithi163 2024-01-27T13:23:40.525 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i -e 's/smithi163/smithi163/g' /etc/hosts 2024-01-27T13:23:40.556 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i -e 's/172.21.15.163/172.21.15.163/g' /etc/hosts 2024-01-27T13:23:40.623 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i -e 's/smithi163/smithi163/g' /etc/hostname 2024-01-27T13:23:40.689 DEBUG:teuthology.orchestra.run.smithi163:> sudo hostname smithi163 2024-01-27T13:23:40.755 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release 2024-01-27T13:23:40.811 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="Red Hat Enterprise Linux" 2024-01-27T13:23:40.811 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="8.6 (Ootpa)" 2024-01-27T13:23:40.811 INFO:teuthology.orchestra.run.smithi163.stdout:ID="rhel" 2024-01-27T13:23:40.812 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="fedora" 2024-01-27T13:23:40.812 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="8.6" 2024-01-27T13:23:40.812 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el8" 2024-01-27T13:23:40.812 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-27T13:23:40.812 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2024-01-27T13:23:40.812 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-27T13:23:40.812 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://www.redhat.com/" 2024-01-27T13:23:40.812 INFO:teuthology.orchestra.run.smithi163.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-27T13:23:40.812 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-27T13:23:40.812 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-27T13:23:40.812 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-27T13:23:40.812 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-27T13:23:40.813 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-27T13:23:40.813 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-27T13:23:40.813 INFO:teuthology.provision.fog.smithi163:Deploy complete! 2024-01-27T13:23:40.814 DEBUG:teuthology.parallel:result is None 2024-01-27T13:23:40.966 INFO:teuthology.lock.ops:Checking smithi119.front.sepia.ceph.com 2024-01-27T13:23:40.966 INFO:teuthology.lock.ops:New key found. Updating... 2024-01-27T13:23:40.993 INFO:teuthology.lock.ops:Checking smithi163.front.sepia.ceph.com 2024-01-27T13:23:40.993 INFO:teuthology.lock.ops:New key found. Updating... 2024-01-27T13:23:41.011 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: set os type and version on server 2024-01-27T13:23:41.011 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-01-27T13:23:41.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:23:41.084 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-01-27T13:23:41.226 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-01-27T13:23:41.226 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2024-01-27T13:23:41.282 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2024-01-27T13:23:41.283 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2024-01-27T13:23:41.341 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Red Hat Enterprise Linux" 2024-01-27T13:23:41.341 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="8.6 (Ootpa)" 2024-01-27T13:23:41.341 INFO:teuthology.orchestra.run.smithi119.stdout:ID="rhel" 2024-01-27T13:23:41.341 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="fedora" 2024-01-27T13:23:41.341 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="8.6" 2024-01-27T13:23:41.341 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el8" 2024-01-27T13:23:41.341 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-27T13:23:41.341 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2024-01-27T13:23:41.341 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-27T13:23:41.341 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.redhat.com/" 2024-01-27T13:23:41.341 INFO:teuthology.orchestra.run.smithi119.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-27T13:23:41.341 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-27T13:23:41.342 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-01-27T13:23:41.342 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-27T13:23:41.342 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-27T13:23:41.342 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-27T13:23:41.342 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-27T13:23:41.342 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2024-01-27T13:23:41.368 INFO:teuthology.lock.ops:Updating [smithi163.front.sepia.ceph.com]: set os type and version on server 2024-01-27T13:23:41.368 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-01-27T13:23:41.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-27T13:23:41.484 DEBUG:teuthology.orchestra.run.smithi163:> true 2024-01-27T13:23:41.551 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-01-27T13:23:41.552 DEBUG:teuthology.orchestra.run.smithi163:> uname -m 2024-01-27T13:23:41.609 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2024-01-27T13:23:41.610 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release 2024-01-27T13:23:41.668 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="Red Hat Enterprise Linux" 2024-01-27T13:23:41.668 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="8.6 (Ootpa)" 2024-01-27T13:23:41.668 INFO:teuthology.orchestra.run.smithi163.stdout:ID="rhel" 2024-01-27T13:23:41.668 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="fedora" 2024-01-27T13:23:41.668 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="8.6" 2024-01-27T13:23:41.668 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el8" 2024-01-27T13:23:41.668 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-27T13:23:41.668 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2024-01-27T13:23:41.668 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-27T13:23:41.669 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://www.redhat.com/" 2024-01-27T13:23:41.669 INFO:teuthology.orchestra.run.smithi163.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-27T13:23:41.669 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-27T13:23:41.669 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-01-27T13:23:41.669 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-27T13:23:41.669 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-27T13:23:41.669 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-27T13:23:41.669 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-27T13:23:41.670 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2024-01-27T13:23:41.693 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-27T13:23:41.782 INFO:teuthology.dispatcher.supervisor:Running job 7534456 2024-01-27T13:23:41.782 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-01-26_15:05:59-fs-main-distro-default-smithi/7534456 --name yuriw-2024-01-26_15:05:59-fs-main-distro-default-smithi --description fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} -- /home/teuthworker/archive/yuriw-2024-01-26_15:05:59-fs-main-distro-default-smithi/7534456/orig.config.yaml 2024-01-27T13:23:41.787 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2024-01-26_15:05:59-fs-main-distro-default-smithi/7534456 2024-01-27T13:23:41.787 INFO:teuthology.dispatcher.supervisor:Job PID: 22905 2024-01-27T13:23:41.787 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-01-27T13:25:41.888 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-27T13:27:42.064 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-27T13:29:42.236 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-27T13:31:42.404 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-27T13:33:42.573 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-27T13:33:42.646 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2024-01-27T13:33:42.777 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2024-01-27T13:33:42.777 INFO:teuthology.nuke:Checking targets against current locks 2024-01-27T13:33:42.830 DEBUG:teuthology.nuke:shortname: smithi119 2024-01-27T13:33:42.830 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-27T13:33:42.852 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-26_15:05:59-fs-main-distro-default-smithi/7534456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-27 13:17:52.396908', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBENP8iusTRr2qnV2pDRlEhjognnI2liXWz7bNzTrZg2F1f0GlpwBXPuy2s/aseEwqRlp1TzDiRLpJ4IdDHj3UeI='} 2024-01-27T13:33:42.855 DEBUG:teuthology.nuke:shortname: smithi163 2024-01-27T13:33:42.855 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-27T13:33:42.882 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-26_15:05:59-fs-main-distro-default-smithi/7534456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-27 13:17:52.394956', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKuuZJvoKjV9iymE7wsEf8ypFJzDg+PTDgxnxsbT/dPYHKeEjtdq28Tw7yoMsypdQWP7XtqyB3Kb5YFJV7YECoQ='} 2024-01-27T13:33:42.911 INFO:teuthology.orchestra.console.smithi119:Power off 2024-01-27T13:33:42.911 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-27T13:33:42.930 INFO:teuthology.orchestra.console.smithi163:Power off 2024-01-27T13:33:42.930 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-27T13:33:42.939 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off 2024-01-27T13:33:42.939 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T13:33:42.952 DEBUG:teuthology.orchestra.console.smithi163:power off output: Chassis Power Control: Down/Off 2024-01-27T13:33:42.952 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-27T13:33:42.963 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off 2024-01-27T13:33:42.975 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off 2024-01-27T13:33:43.064 INFO:teuthology.orchestra.console.smithi119:Power off completed 2024-01-27T13:33:43.076 INFO:teuthology.orchestra.console.smithi163:Power off completed 2024-01-27T13:33:43.218 INFO:teuthology.lock.ops:unlocked: smithi119.front.sepia.ceph.com 2024-01-27T13:33:43.226 INFO:teuthology.lock.ops:unlocked: smithi163.front.sepia.ceph.com