2023-11-10T22:07:54.666 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-10T22:07:54.668 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T22:07:54.802 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd431210670>, signals=[15]) 2023-11-10T22:07:54.803 INFO:teuthology.lock.ops:Start node 'smithi087.front.sepia.ceph.com' reimaging 2023-11-10T22:07:54.803 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: reset os type and version on server 2023-11-10T22:07:54.804 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2023-11-10T22:07:54.825 INFO:teuthology.lock.ops:Start node 'smithi148.front.sepia.ceph.com' reimaging 2023-11-10T22:07:54.826 INFO:teuthology.lock.ops:Updating [smithi148.front.sepia.ceph.com]: reset os type and version on server 2023-11-10T22:07:54.826 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2023-11-10T22:07:54.828 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /host {"name": "smithi087"} 2023-11-10T22:07:54.850 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /host {"name": "smithi148"} 2023-11-10T22:07:54.900 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-11-10T22:07:54.924 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-11-10T22:07:54.941 DEBUG:teuthology.provision.fog.smithi087:Requesting image smithi_centos_9.stream (ID 10) 2023-11-10T22:07:54.941 DEBUG:teuthology.provision.fog.smithi087:FOG request: PUT /host/192 {"imageID": 10} 2023-11-10T22:07:54.965 DEBUG:teuthology.provision.fog.smithi148:Requesting image smithi_centos_9.stream (ID 10) 2023-11-10T22:07:54.965 DEBUG:teuthology.provision.fog.smithi148:FOG request: PUT /host/253 {"imageID": 10} 2023-11-10T22:07:54.992 INFO:teuthology.provision.fog.smithi087:Scheduling deploy of centos 9.stream 2023-11-10T22:07:54.992 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-11-10T22:07:55.008 INFO:teuthology.provision.fog.smithi148:Scheduling deploy of centos 9.stream 2023-11-10T22:07:55.008 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-10T22:07:55.048 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /tasktype {"name": "deploy"} 2023-11-10T22:07:55.066 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /tasktype {"name": "deploy"} 2023-11-10T22:07:55.079 DEBUG:teuthology.provision.fog.smithi087:FOG request: POST /host/192/task {"taskTypeID": 1} 2023-11-10T22:07:55.102 DEBUG:teuthology.provision.fog.smithi148:FOG request: POST /host/253/task {"taskTypeID": 1} 2023-11-10T22:07:55.152 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-11-10T22:07:55.170 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-10T22:07:55.249 INFO:teuthology.orchestra.console.smithi087:Power off 2023-11-10T22:07:55.250 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-10T22:07:55.269 INFO:teuthology.orchestra.console.smithi148:Power off 2023-11-10T22:07:55.272 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-10T22:07:55.282 DEBUG:teuthology.orchestra.console.smithi087:power off output: Chassis Power Control: Down/Off 2023-11-10T22:07:55.283 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T22:07:55.296 DEBUG:teuthology.orchestra.console.smithi148:power off output: Chassis Power Control: Down/Off 2023-11-10T22:07:55.297 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T22:07:55.308 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2023-11-10T22:07:55.322 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-11-10T22:07:59.310 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T22:07:59.324 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T22:07:59.421 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2023-11-10T22:07:59.438 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-11-10T22:08:03.422 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T22:08:03.440 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T22:08:03.532 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2023-11-10T22:08:03.553 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-11-10T22:08:07.534 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T22:08:07.554 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T22:08:07.644 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2023-11-10T22:08:07.664 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2023-11-10T22:08:07.746 INFO:teuthology.orchestra.console.smithi087:Power off completed 2023-11-10T22:08:07.765 INFO:teuthology.orchestra.console.smithi148:Power off completed 2023-11-10T22:08:07.849 INFO:teuthology.orchestra.console.smithi087:Power on 2023-11-10T22:08:07.850 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-10T22:08:07.866 INFO:teuthology.orchestra.console.smithi148:Power on 2023-11-10T22:08:07.866 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-10T22:08:07.877 DEBUG:teuthology.orchestra.console.smithi087:power on output: Chassis Power Control: Up/On 2023-11-10T22:08:07.878 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T22:08:07.893 DEBUG:teuthology.orchestra.console.smithi148:power on output: Chassis Power Control: Up/On 2023-11-10T22:08:07.894 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T22:08:07.904 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2023-11-10T22:08:07.917 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2023-11-10T22:08:11.907 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T22:08:11.918 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T22:08:12.020 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2023-11-10T22:08:12.027 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-11-10T22:08:12.121 INFO:teuthology.orchestra.console.smithi087:Power on completed 2023-11-10T22:08:12.131 INFO:teuthology.orchestra.console.smithi148:Power on completed 2023-11-10T22:08:12.222 INFO:teuthology.provision.fog.smithi087:Waiting for deploy to finish 2023-11-10T22:08:12.223 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-11-10T22:08:12.232 INFO:teuthology.provision.fog.smithi148:Waiting for deploy to finish 2023-11-10T22:08:12.232 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-10T22:08:27.303 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-11-10T22:08:27.305 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-10T22:08:42.388 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-10T22:08:42.390 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-11-10T22:08:57.470 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-11-10T22:08:57.472 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-10T22:09:12.562 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-11-10T22:09:12.563 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-10T22:09:27.650 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-10T22:09:27.652 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-11-10T22:09:42.743 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-11-10T22:09:42.745 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-10T22:09:57.846 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-10T22:09:57.847 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-11-10T22:10:12.966 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-11-10T22:10:12.968 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-10T22:10:28.070 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-10T22:10:28.072 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-11-10T22:10:43.150 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-10T22:10:43.152 DEBUG:teuthology.provision.fog.smithi087:FOG request: GET /task/active None 2023-11-10T22:10:43.216 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:10:46.301 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-11-10T22:10:52.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:10:55.386 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-11-10T22:10:58.234 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-11-10T22:10:58.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:11:01.370 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-10T22:11:01.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:11:04.446 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-11-10T22:11:07.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:11:10.431 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-10T22:11:10.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:11:13.502 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-11-10T22:11:16.434 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:11:19.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:11:19.514 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-10T22:11:22.590 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-11-10T22:11:25.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:11:28.573 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-10T22:11:28.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:11:31.643 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-11-10T22:11:34.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:11:37.648 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:11:37.662 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-10T22:11:40.698 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-11-10T22:11:43.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:11:46.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:11:46.746 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-10T22:11:52.750 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:11:55.804 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-11-10T22:12:01.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:12:46.750 WARNING:teuthology.provision.fog.smithi087:timed out 2023-11-10T22:12:52.757 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:12:53.085 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-10T22:12:53.086 DEBUG:teuthology.orchestra.run.smithi087:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-10T22:12:53.396 INFO:teuthology.provision.fog.smithi087:Node is ready 2023-11-10T22:12:53.396 DEBUG:teuthology.orchestra.run.smithi087:> hostname 2023-11-10T22:12:53.411 INFO:teuthology.orchestra.run.smithi087.stdout:smithi087.front.sepia.ceph.com 2023-11-10T22:12:53.412 DEBUG:teuthology.orchestra.run.smithi087:> grep smithi087.front.sepia.ceph.com /etc/hosts 2023-11-10T22:12:53.470 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87 smithi087.front.sepia.ceph.com smithi087 2023-11-10T22:12:53.470 DEBUG:teuthology.orchestra.run.smithi087:> sudo hostname smithi087 2023-11-10T22:12:53.583 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/smithi087.front.sepia.ceph.com/smithi087/g' /etc/hosts 2023-11-10T22:12:53.611 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/172.21.15.87/172.21.15.87/g' /etc/hosts 2023-11-10T22:12:53.682 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/smithi087.front.sepia.ceph.com/smithi087/g' /etc/hostname 2023-11-10T22:12:53.754 DEBUG:teuthology.orchestra.run.smithi087:> sudo hostname smithi087 2023-11-10T22:12:53.823 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2023-11-10T22:12:53.886 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="CentOS Stream" 2023-11-10T22:12:53.886 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="9" 2023-11-10T22:12:53.886 INFO:teuthology.orchestra.run.smithi087.stdout:ID="centos" 2023-11-10T22:12:53.886 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="rhel fedora" 2023-11-10T22:12:53.887 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="9" 2023-11-10T22:12:53.887 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el9" 2023-11-10T22:12:53.887 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-10T22:12:53.887 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2023-11-10T22:12:53.887 INFO:teuthology.orchestra.run.smithi087.stdout:LOGO="fedora-logo-icon" 2023-11-10T22:12:53.888 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-10T22:12:53.888 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://centos.org/" 2023-11-10T22:12:53.888 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-10T22:12:53.888 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-10T22:12:53.888 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-10T22:12:53.889 INFO:teuthology.provision.fog.smithi087:Deploy complete! 2023-11-10T22:12:53.890 DEBUG:teuthology.parallel:result is None 2023-11-10T22:13:01.818 WARNING:teuthology.provision.fog.smithi148:timed out 2023-11-10T22:13:07.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:13:08.183 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-10T22:13:08.184 DEBUG:teuthology.orchestra.run.smithi148:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-10T22:13:08.528 INFO:teuthology.provision.fog.smithi148:Node is ready 2023-11-10T22:13:08.528 DEBUG:teuthology.orchestra.run.smithi148:> hostname 2023-11-10T22:13:08.542 INFO:teuthology.orchestra.run.smithi148.stdout:smithi148.front.sepia.ceph.com 2023-11-10T22:13:08.543 DEBUG:teuthology.orchestra.run.smithi148:> grep smithi148.front.sepia.ceph.com /etc/hosts 2023-11-10T22:13:08.603 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.15.148 smithi148.front.sepia.ceph.com smithi148 2023-11-10T22:13:08.603 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname smithi148 2023-11-10T22:13:08.722 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i -e 's/smithi148.front.sepia.ceph.com/smithi148/g' /etc/hosts 2023-11-10T22:13:08.749 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i -e 's/172.21.15.148/172.21.15.148/g' /etc/hosts 2023-11-10T22:13:08.820 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i -e 's/smithi148.front.sepia.ceph.com/smithi148/g' /etc/hostname 2023-11-10T22:13:08.891 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname smithi148 2023-11-10T22:13:08.961 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2023-11-10T22:13:09.019 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2023-11-10T22:13:09.020 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2023-11-10T22:13:09.020 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2023-11-10T22:13:09.020 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2023-11-10T22:13:09.020 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2023-11-10T22:13:09.021 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2023-11-10T22:13:09.021 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-10T22:13:09.021 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2023-11-10T22:13:09.021 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2023-11-10T22:13:09.021 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-10T22:13:09.022 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2023-11-10T22:13:09.022 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-10T22:13:09.022 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-10T22:13:09.022 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-10T22:13:09.023 INFO:teuthology.provision.fog.smithi148:Deploy complete! 2023-11-10T22:13:09.023 DEBUG:teuthology.parallel:result is None 2023-11-10T22:13:09.205 INFO:teuthology.lock.ops:Checking smithi087.front.sepia.ceph.com 2023-11-10T22:13:09.206 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-10T22:13:09.232 INFO:teuthology.lock.ops:Checking smithi148.front.sepia.ceph.com 2023-11-10T22:13:09.233 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-10T22:13:09.258 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: set os type and version on server 2023-11-10T22:13:09.259 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-11-10T22:13:09.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:13:09.344 DEBUG:teuthology.orchestra.run.smithi087:> true 2023-11-10T22:13:09.512 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-11-10T22:13:09.513 DEBUG:teuthology.orchestra.run.smithi087:> uname -m 2023-11-10T22:13:09.567 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2023-11-10T22:13:09.568 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2023-11-10T22:13:09.622 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="CentOS Stream" 2023-11-10T22:13:09.622 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="9" 2023-11-10T22:13:09.622 INFO:teuthology.orchestra.run.smithi087.stdout:ID="centos" 2023-11-10T22:13:09.622 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="rhel fedora" 2023-11-10T22:13:09.623 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="9" 2023-11-10T22:13:09.623 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el9" 2023-11-10T22:13:09.624 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-10T22:13:09.624 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2023-11-10T22:13:09.625 INFO:teuthology.orchestra.run.smithi087.stdout:LOGO="fedora-logo-icon" 2023-11-10T22:13:09.625 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-10T22:13:09.625 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://centos.org/" 2023-11-10T22:13:09.625 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-10T22:13:09.625 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-10T22:13:09.626 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-10T22:13:09.626 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2023-11-10T22:13:09.649 INFO:teuthology.lock.ops:Updating [smithi148.front.sepia.ceph.com]: set os type and version on server 2023-11-10T22:13:09.649 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-11-10T22:13:09.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-10T22:13:09.776 DEBUG:teuthology.orchestra.run.smithi148:> true 2023-11-10T22:13:09.844 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-11-10T22:13:09.845 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2023-11-10T22:13:09.902 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2023-11-10T22:13:09.902 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2023-11-10T22:13:09.956 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2023-11-10T22:13:09.956 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2023-11-10T22:13:09.956 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2023-11-10T22:13:09.957 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2023-11-10T22:13:09.957 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2023-11-10T22:13:09.957 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2023-11-10T22:13:09.957 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-10T22:13:09.957 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2023-11-10T22:13:09.958 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2023-11-10T22:13:09.958 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-10T22:13:09.958 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2023-11-10T22:13:09.958 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-10T22:13:09.959 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-10T22:13:09.959 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-10T22:13:09.959 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2023-11-10T22:13:09.982 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T22:13:10.073 INFO:teuthology.dispatcher.supervisor:Running job 7455568 2023-11-10T22:13:10.073 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_aemerson@teuthology --archive /home/teuthworker/archive/aemerson-2023-11-10_22:02:58-rbd-main-distro-default-smithi/7455568 --name aemerson-2023-11-10_22:02:58-rbd-main-distro-default-smithi --description rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} -- /home/teuthworker/archive/aemerson-2023-11-10_22:02:58-rbd-main-distro-default-smithi/7455568/orig.config.yaml 2023-11-10T22:13:10.081 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/aemerson-2023-11-10_22:02:58-rbd-main-distro-default-smithi/7455568 2023-11-10T22:13:10.082 INFO:teuthology.dispatcher.supervisor:Job PID: 1346414 2023-11-10T22:13:10.082 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-10T22:15:10.122 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T22:17:10.295 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T22:19:10.467 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T22:21:10.635 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T22:23:10.803 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T22:25:10.971 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T22:27:11.128 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T22:29:11.289 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T22:31:11.451 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-10T22:31:11.516 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-11-10T22:31:11.630 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-10T22:31:11.630 INFO:teuthology.nuke:Checking targets against current locks 2023-11-10T22:31:11.686 DEBUG:teuthology.nuke:shortname: smithi087 2023-11-10T22:31:11.687 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-10T22:31:11.701 DEBUG:teuthology.nuke:shortname: smithi148 2023-11-10T22:31:11.702 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-10T22:31:11.706 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/aemerson-2023-11-10_22:02:58-rbd-main-distro-default-smithi/7455568', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-10 22:07:53.836293', 'locked_by': 'scheduled_aemerson@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOjxGu5w+eReL23tU0soVVkIiOi5hyj2LAMA7qLDAsFCOLyJO4NTcsC5HTMFFcbUjn0gfaeKdC4AddoLE+vqBiQ='} 2023-11-10T22:31:11.732 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/aemerson-2023-11-10_22:02:58-rbd-main-distro-default-smithi/7455568', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-10 22:07:53.834830', 'locked_by': 'scheduled_aemerson@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIfnTipgFZI5LqEFOEzDG9qfYlzZ9QDYITtrYgKiXn/vYKMsDdc5Be2SZikQ3cvHPUM+rkQci8P/j7l/t9niXPs='} 2023-11-10T22:31:11.771 INFO:teuthology.orchestra.console.smithi087:Power off 2023-11-10T22:31:11.772 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-10T22:31:11.796 DEBUG:teuthology.orchestra.console.smithi087:power off output: Chassis Power Control: Down/Off 2023-11-10T22:31:11.797 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T22:31:11.813 INFO:teuthology.orchestra.console.smithi148:Power off 2023-11-10T22:31:11.814 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-10T22:31:11.826 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2023-11-10T22:31:11.841 DEBUG:teuthology.orchestra.console.smithi148:power off output: Chassis Power Control: Down/Off 2023-11-10T22:31:11.841 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-10T22:31:11.869 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2023-11-10T22:31:11.927 INFO:teuthology.orchestra.console.smithi087:Power off completed 2023-11-10T22:31:11.971 INFO:teuthology.orchestra.console.smithi148:Power off completed 2023-11-10T22:31:12.076 INFO:teuthology.lock.ops:unlocked: smithi087.front.sepia.ceph.com 2023-11-10T22:31:12.115 INFO:teuthology.lock.ops:unlocked: smithi148.front.sepia.ceph.com