2023-11-14T06:12:14.720 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-14T06:12:14.722 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:12:14.867 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7feb5f8f9670>, signals=[15]) 2023-11-14T06:12:14.868 INFO:teuthology.lock.ops:Start node 'smithi176.front.sepia.ceph.com' reimaging 2023-11-14T06:12:14.869 INFO:teuthology.lock.ops:Updating [smithi176.front.sepia.ceph.com]: reset os type and version on server 2023-11-14T06:12:14.869 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2023-11-14T06:12:14.886 INFO:teuthology.lock.ops:Start node 'smithi188.front.sepia.ceph.com' reimaging 2023-11-14T06:12:14.887 INFO:teuthology.lock.ops:Updating [smithi188.front.sepia.ceph.com]: reset os type and version on server 2023-11-14T06:12:14.887 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2023-11-14T06:12:14.888 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /host {"name": "smithi176"} 2023-11-14T06:12:14.911 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /host {"name": "smithi188"} 2023-11-14T06:12:14.967 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-11-14T06:12:14.980 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-11-14T06:12:15.025 DEBUG:teuthology.provision.fog.smithi188:Requesting image smithi_centos_9.stream (ID 10) 2023-11-14T06:12:15.026 DEBUG:teuthology.provision.fog.smithi188:FOG request: PUT /host/293 {"imageID": 10} 2023-11-14T06:12:15.029 DEBUG:teuthology.provision.fog.smithi176:Requesting image smithi_centos_9.stream (ID 10) 2023-11-14T06:12:15.029 DEBUG:teuthology.provision.fog.smithi176:FOG request: PUT /host/281 {"imageID": 10} 2023-11-14T06:12:15.078 INFO:teuthology.provision.fog.smithi188:Scheduling deploy of centos 9.stream 2023-11-14T06:12:15.079 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-11-14T06:12:15.082 INFO:teuthology.provision.fog.smithi176:Scheduling deploy of centos 9.stream 2023-11-14T06:12:15.082 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /task/active None 2023-11-14T06:12:15.182 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /tasktype {"name": "deploy"} 2023-11-14T06:12:15.187 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /tasktype {"name": "deploy"} 2023-11-14T06:12:15.233 DEBUG:teuthology.provision.fog.smithi176:FOG request: POST /host/281/task {"taskTypeID": 1} 2023-11-14T06:12:15.237 DEBUG:teuthology.provision.fog.smithi188:FOG request: POST /host/293/task {"taskTypeID": 1} 2023-11-14T06:12:15.306 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /task/active None 2023-11-14T06:12:15.309 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-11-14T06:12:15.459 INFO:teuthology.orchestra.console.smithi176:Power off 2023-11-14T06:12:15.460 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-14T06:12:15.473 INFO:teuthology.orchestra.console.smithi188:Power off 2023-11-14T06:12:15.473 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-14T06:12:15.483 DEBUG:teuthology.orchestra.console.smithi176:power off output: Chassis Power Control: Down/Off 2023-11-14T06:12:15.484 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:12:15.498 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off 2023-11-14T06:12:15.499 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:12:15.508 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is on 2023-11-14T06:12:15.520 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2023-11-14T06:12:19.510 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:12:19.521 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:12:19.621 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is on 2023-11-14T06:12:19.631 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2023-11-14T06:12:23.626 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:12:23.633 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:12:23.735 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is on 2023-11-14T06:12:23.742 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2023-11-14T06:12:23.843 INFO:teuthology.orchestra.console.smithi188:Power off completed 2023-11-14T06:12:23.944 INFO:teuthology.orchestra.console.smithi188:Power on 2023-11-14T06:12:23.944 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-14T06:12:23.968 DEBUG:teuthology.orchestra.console.smithi188:power on output: Chassis Power Control: Up/On 2023-11-14T06:12:23.969 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:12:23.990 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2023-11-14T06:12:27.738 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:12:27.847 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is off 2023-11-14T06:12:27.948 INFO:teuthology.orchestra.console.smithi176:Power off completed 2023-11-14T06:12:27.991 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:12:28.050 INFO:teuthology.orchestra.console.smithi176:Power on 2023-11-14T06:12:28.050 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-14T06:12:28.072 DEBUG:teuthology.orchestra.console.smithi176:power on output: Chassis Power Control: Up/On 2023-11-14T06:12:28.073 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:12:28.099 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is off 2023-11-14T06:12:28.101 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2023-11-14T06:12:32.102 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:12:32.111 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T06:12:32.213 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is on 2023-11-14T06:12:32.219 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2023-11-14T06:12:32.314 INFO:teuthology.orchestra.console.smithi176:Power on completed 2023-11-14T06:12:32.321 INFO:teuthology.orchestra.console.smithi188:Power on completed 2023-11-14T06:12:32.417 INFO:teuthology.provision.fog.smithi176:Waiting for deploy to finish 2023-11-14T06:12:32.417 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /task/active None 2023-11-14T06:12:32.422 INFO:teuthology.provision.fog.smithi188:Waiting for deploy to finish 2023-11-14T06:12:32.422 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-11-14T06:12:47.530 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /task/active None 2023-11-14T06:12:47.533 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-11-14T06:13:02.644 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /task/active None 2023-11-14T06:13:02.646 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-11-14T06:13:17.758 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /task/active None 2023-11-14T06:13:17.760 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-11-14T06:13:32.873 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /task/active None 2023-11-14T06:13:32.876 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-11-14T06:13:49.218 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /task/active None 2023-11-14T06:13:49.219 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-11-14T06:14:04.370 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-11-14T06:14:04.379 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /task/active None 2023-11-14T06:14:19.505 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-11-14T06:14:19.506 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /task/active None 2023-11-14T06:14:34.642 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /task/active None 2023-11-14T06:14:34.649 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-11-14T06:14:49.770 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /task/active None 2023-11-14T06:14:49.771 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-11-14T06:15:04.914 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-11-14T06:15:04.916 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /task/active None 2023-11-14T06:15:20.018 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /task/active None 2023-11-14T06:15:20.020 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-11-14T06:15:20.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:15:23.166 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-11-14T06:15:29.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:15:32.250 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-11-14T06:15:35.106 DEBUG:teuthology.provision.fog.smithi176:FOG request: GET /task/active None 2023-11-14T06:15:35.180 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:15:38.238 WARNING:teuthology.provision.fog.smithi176:[Errno None] Unable to connect to port 22 on 172.21.15.176 2023-11-14T06:15:38.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:15:41.306 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-11-14T06:15:44.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:15:47.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:15:47.322 WARNING:teuthology.provision.fog.smithi176:[Errno None] Unable to connect to port 22 on 172.21.15.176 2023-11-14T06:15:50.362 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-11-14T06:15:53.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:15:56.366 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:15:56.378 WARNING:teuthology.provision.fog.smithi176:[Errno None] Unable to connect to port 22 on 172.21.15.176 2023-11-14T06:15:59.418 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-11-14T06:16:02.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:16:05.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:16:05.434 WARNING:teuthology.provision.fog.smithi176:[Errno None] Unable to connect to port 22 on 172.21.15.176 2023-11-14T06:16:08.478 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-11-14T06:16:11.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:16:14.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:16:14.490 WARNING:teuthology.provision.fog.smithi176:[Errno None] Unable to connect to port 22 on 172.21.15.176 2023-11-14T06:16:17.562 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-11-14T06:16:20.494 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:16:23.550 WARNING:teuthology.provision.fog.smithi176:[Errno None] Unable to connect to port 22 on 172.21.15.176 2023-11-14T06:16:23.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:16:29.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:16:42.110 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-11-14T06:16:48.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:16:51.194 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-11-14T06:16:57.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:17:00.250 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-11-14T06:17:06.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:17:06.548 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-14T06:17:06.549 DEBUG:teuthology.orchestra.run.smithi188:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-14T06:17:22.110 INFO:teuthology.provision.fog.smithi188:Node is ready 2023-11-14T06:17:22.110 DEBUG:teuthology.orchestra.run.smithi188:> hostname 2023-11-14T06:17:22.132 INFO:teuthology.orchestra.run.smithi188.stdout:smithi188.front.sepia.ceph.com 2023-11-14T06:17:22.133 DEBUG:teuthology.orchestra.run.smithi188:> grep smithi188.front.sepia.ceph.com /etc/hosts 2023-11-14T06:17:22.188 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.15.188 smithi188.front.sepia.ceph.com smithi188 2023-11-14T06:17:22.189 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname smithi188 2023-11-14T06:17:22.311 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i -e 's/smithi188.front.sepia.ceph.com/smithi188/g' /etc/hosts 2023-11-14T06:17:22.340 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i -e 's/172.21.15.188/172.21.15.188/g' /etc/hosts 2023-11-14T06:17:22.422 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i -e 's/smithi188.front.sepia.ceph.com/smithi188/g' /etc/hostname 2023-11-14T06:17:22.495 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname smithi188 2023-11-14T06:17:22.566 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2023-11-14T06:17:22.627 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2023-11-14T06:17:22.627 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2023-11-14T06:17:22.627 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2023-11-14T06:17:22.628 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2023-11-14T06:17:22.628 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2023-11-14T06:17:22.628 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2023-11-14T06:17:22.628 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-14T06:17:22.628 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2023-11-14T06:17:22.629 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2023-11-14T06:17:22.629 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-14T06:17:22.629 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2023-11-14T06:17:22.629 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-14T06:17:22.629 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-14T06:17:22.630 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-14T06:17:22.631 INFO:teuthology.provision.fog.smithi188:Deploy complete! 2023-11-14T06:17:22.632 DEBUG:teuthology.parallel:result is None 2023-11-14T06:17:29.564 WARNING:teuthology.provision.fog.smithi176:timed out 2023-11-14T06:17:35.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:17:35.791 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-14T06:17:35.791 DEBUG:teuthology.orchestra.run.smithi176:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-14T06:17:36.124 INFO:teuthology.provision.fog.smithi176:Node is ready 2023-11-14T06:17:36.124 DEBUG:teuthology.orchestra.run.smithi176:> hostname 2023-11-14T06:17:36.139 INFO:teuthology.orchestra.run.smithi176.stdout:smithi176.front.sepia.ceph.com 2023-11-14T06:17:36.139 DEBUG:teuthology.orchestra.run.smithi176:> grep smithi176.front.sepia.ceph.com /etc/hosts 2023-11-14T06:17:36.194 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176 smithi176.front.sepia.ceph.com smithi176 2023-11-14T06:17:36.195 DEBUG:teuthology.orchestra.run.smithi176:> sudo hostname smithi176 2023-11-14T06:17:36.304 DEBUG:teuthology.orchestra.run.smithi176:> sudo sed -i -e 's/smithi176.front.sepia.ceph.com/smithi176/g' /etc/hosts 2023-11-14T06:17:36.333 DEBUG:teuthology.orchestra.run.smithi176:> sudo sed -i -e 's/172.21.15.176/172.21.15.176/g' /etc/hosts 2023-11-14T06:17:36.400 DEBUG:teuthology.orchestra.run.smithi176:> sudo sed -i -e 's/smithi176.front.sepia.ceph.com/smithi176/g' /etc/hostname 2023-11-14T06:17:36.467 DEBUG:teuthology.orchestra.run.smithi176:> sudo hostname smithi176 2023-11-14T06:17:36.533 DEBUG:teuthology.orchestra.run.smithi176:> cat /etc/os-release 2023-11-14T06:17:36.587 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="CentOS Stream" 2023-11-14T06:17:36.588 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="9" 2023-11-14T06:17:36.588 INFO:teuthology.orchestra.run.smithi176.stdout:ID="centos" 2023-11-14T06:17:36.588 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE="rhel fedora" 2023-11-14T06:17:36.588 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="9" 2023-11-14T06:17:36.589 INFO:teuthology.orchestra.run.smithi176.stdout:PLATFORM_ID="platform:el9" 2023-11-14T06:17:36.589 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-14T06:17:36.589 INFO:teuthology.orchestra.run.smithi176.stdout:ANSI_COLOR="0;31" 2023-11-14T06:17:36.589 INFO:teuthology.orchestra.run.smithi176.stdout:LOGO="fedora-logo-icon" 2023-11-14T06:17:36.590 INFO:teuthology.orchestra.run.smithi176.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-14T06:17:36.590 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://centos.org/" 2023-11-14T06:17:36.590 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-14T06:17:36.590 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-14T06:17:36.591 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-14T06:17:36.591 INFO:teuthology.provision.fog.smithi176:Deploy complete! 2023-11-14T06:17:36.592 DEBUG:teuthology.parallel:result is None 2023-11-14T06:17:36.767 INFO:teuthology.lock.ops:Checking smithi176.front.sepia.ceph.com 2023-11-14T06:17:36.767 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-14T06:17:36.791 INFO:teuthology.lock.ops:Checking smithi188.front.sepia.ceph.com 2023-11-14T06:17:36.792 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-14T06:17:36.818 INFO:teuthology.lock.ops:Updating [smithi176.front.sepia.ceph.com]: set os type and version on server 2023-11-14T06:17:36.819 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2023-11-14T06:17:36.820 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:17:36.904 DEBUG:teuthology.orchestra.run.smithi176:> true 2023-11-14T06:17:36.972 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi176.front.sepia.ceph.com' 2023-11-14T06:17:36.972 DEBUG:teuthology.orchestra.run.smithi176:> uname -m 2023-11-14T06:17:37.027 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64 2023-11-14T06:17:37.028 DEBUG:teuthology.orchestra.run.smithi176:> cat /etc/os-release 2023-11-14T06:17:37.081 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="CentOS Stream" 2023-11-14T06:17:37.082 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="9" 2023-11-14T06:17:37.082 INFO:teuthology.orchestra.run.smithi176.stdout:ID="centos" 2023-11-14T06:17:37.082 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE="rhel fedora" 2023-11-14T06:17:37.083 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="9" 2023-11-14T06:17:37.083 INFO:teuthology.orchestra.run.smithi176.stdout:PLATFORM_ID="platform:el9" 2023-11-14T06:17:37.083 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-14T06:17:37.083 INFO:teuthology.orchestra.run.smithi176.stdout:ANSI_COLOR="0;31" 2023-11-14T06:17:37.083 INFO:teuthology.orchestra.run.smithi176.stdout:LOGO="fedora-logo-icon" 2023-11-14T06:17:37.084 INFO:teuthology.orchestra.run.smithi176.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-14T06:17:37.084 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://centos.org/" 2023-11-14T06:17:37.084 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-14T06:17:37.084 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-14T06:17:37.084 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-14T06:17:37.085 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2023-11-14T06:17:37.116 INFO:teuthology.lock.ops:Updating [smithi188.front.sepia.ceph.com]: set os type and version on server 2023-11-14T06:17:37.117 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-11-14T06:17:37.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-14T06:17:37.199 DEBUG:teuthology.orchestra.run.smithi188:> true 2023-11-14T06:17:37.368 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-11-14T06:17:37.369 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2023-11-14T06:17:37.424 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2023-11-14T06:17:37.424 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2023-11-14T06:17:37.479 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2023-11-14T06:17:37.479 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2023-11-14T06:17:37.479 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2023-11-14T06:17:37.479 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2023-11-14T06:17:37.480 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2023-11-14T06:17:37.480 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2023-11-14T06:17:37.480 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-14T06:17:37.480 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2023-11-14T06:17:37.480 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2023-11-14T06:17:37.481 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-14T06:17:37.481 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2023-11-14T06:17:37.481 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-14T06:17:37.481 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-14T06:17:37.482 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-14T06:17:37.483 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2023-11-14T06:17:37.508 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:17:37.607 INFO:teuthology.dispatcher.supervisor:Running job 7456863 2023-11-14T06:17:37.607 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-11-14_06:10:15-fs-wip-vshankar-testing-20231107.042705-distro-default-smithi/7456863 --name vshankar-2023-11-14_06:10:15-fs-wip-vshankar-testing-20231107.042705-distro-default-smithi --description fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/dbench validater/valgrind} -- /home/teuthworker/archive/vshankar-2023-11-14_06:10:15-fs-wip-vshankar-testing-20231107.042705-distro-default-smithi/7456863/orig.config.yaml 2023-11-14T06:17:37.611 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2023-11-14_06:10:15-fs-wip-vshankar-testing-20231107.042705-distro-default-smithi/7456863 2023-11-14T06:17:37.611 INFO:teuthology.dispatcher.supervisor:Job PID: 2310917 2023-11-14T06:17:37.612 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-14T06:19:37.709 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:21:37.871 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:23:38.028 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:25:38.197 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:27:38.363 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:29:38.530 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:31:38.689 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:33:38.851 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:35:39.019 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:37:39.187 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:39:39.359 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:41:39.525 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:43:39.651 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:45:39.811 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:47:39.970 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:49:40.131 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:51:40.289 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:53:40.443 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:55:40.595 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:57:40.763 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T06:59:40.936 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:01:41.119 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:03:41.282 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:05:41.411 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:07:41.568 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-14T07:07:41.631 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-11-14T07:07:41.804 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-14T07:07:41.804 INFO:teuthology.nuke:Checking targets against current locks 2023-11-14T07:07:41.855 DEBUG:teuthology.nuke:shortname: smithi176 2023-11-14T07:07:41.856 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-14T07:07:41.878 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi176.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-14_06:10:15-fs-wip-vshankar-testing-20231107.042705-distro-default-smithi/7456863', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-14 06:12:13.887893', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD9QSy8BhJBSp38ERDT8LNJalh9uV/y+0MN9+/eNhqiod1O+RgVpco3oy9P8jaTRcoZ8GABoPAv3fLoj71cSa+U='} 2023-11-14T07:07:41.881 DEBUG:teuthology.nuke:shortname: smithi188 2023-11-14T07:07:41.881 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-14T07:07:41.907 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-14_06:10:15-fs-wip-vshankar-testing-20231107.042705-distro-default-smithi/7456863', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-14 06:12:13.886193', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG5etc6B6xJp6EkNB4AWEwocyxN9Ax+vbKd1ux6UJ1iXS3fWL1jaiYxRtrDp4Kl/D6M7o8BjPQwk9KVr8+WoYB4='} 2023-11-14T07:07:41.936 INFO:teuthology.orchestra.console.smithi176:Power off 2023-11-14T07:07:41.936 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-14T07:07:41.951 INFO:teuthology.orchestra.console.smithi188:Power off 2023-11-14T07:07:41.952 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-14T07:07:41.960 DEBUG:teuthology.orchestra.console.smithi176:power off output: Chassis Power Control: Down/Off 2023-11-14T07:07:41.961 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T07:07:41.973 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off 2023-11-14T07:07:41.974 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-14T07:07:41.981 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is off 2023-11-14T07:07:41.993 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2023-11-14T07:07:42.082 INFO:teuthology.orchestra.console.smithi176:Power off completed 2023-11-14T07:07:42.094 INFO:teuthology.orchestra.console.smithi188:Power off completed 2023-11-14T07:07:42.237 INFO:teuthology.lock.ops:unlocked: smithi176.front.sepia.ceph.com 2023-11-14T07:07:42.245 INFO:teuthology.lock.ops:unlocked: smithi188.front.sepia.ceph.com