2023-10-06T10:37:15.904 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T10:37:15.904 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T10:37:16.063 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff0af367670>, signals=[15]) 2023-10-06T10:37:16.063 INFO:teuthology.lock.ops:Start node 'smithi088.front.sepia.ceph.com' reimaging 2023-10-06T10:37:16.064 INFO:teuthology.lock.ops:Updating [smithi088.front.sepia.ceph.com]: reset os type and version on server 2023-10-06T10:37:16.064 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2023-10-06T10:37:16.091 INFO:teuthology.lock.ops:Start node 'smithi131.front.sepia.ceph.com' reimaging 2023-10-06T10:37:16.091 INFO:teuthology.lock.ops:Updating [smithi131.front.sepia.ceph.com]: reset os type and version on server 2023-10-06T10:37:16.091 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2023-10-06T10:37:16.092 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /host {"name": "smithi088"} 2023-10-06T10:37:16.115 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /host {"name": "smithi131"} 2023-10-06T10:37:16.173 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-10-06T10:37:16.180 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-10-06T10:37:16.212 DEBUG:teuthology.provision.fog.smithi088:Requesting image smithi_centos_8.stream (ID 3) 2023-10-06T10:37:16.212 DEBUG:teuthology.provision.fog.smithi088:FOG request: PUT /host/193 {"imageID": 3} 2023-10-06T10:37:16.220 DEBUG:teuthology.provision.fog.smithi131:Requesting image smithi_centos_8.stream (ID 3) 2023-10-06T10:37:16.221 DEBUG:teuthology.provision.fog.smithi131:FOG request: PUT /host/236 {"imageID": 3} 2023-10-06T10:37:16.259 INFO:teuthology.provision.fog.smithi088:Scheduling deploy of centos 8.stream 2023-10-06T10:37:16.260 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /task/active None 2023-10-06T10:37:16.263 INFO:teuthology.provision.fog.smithi131:Scheduling deploy of centos 8.stream 2023-10-06T10:37:16.263 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-06T10:37:16.305 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /tasktype {"name": "deploy"} 2023-10-06T10:37:16.317 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /tasktype {"name": "deploy"} 2023-10-06T10:37:16.347 DEBUG:teuthology.provision.fog.smithi088:FOG request: POST /host/193/task {"taskTypeID": 1} 2023-10-06T10:37:16.352 DEBUG:teuthology.provision.fog.smithi131:FOG request: POST /host/236/task {"taskTypeID": 1} 2023-10-06T10:37:16.404 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /task/active None 2023-10-06T10:37:16.407 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-06T10:37:16.489 INFO:teuthology.orchestra.console.smithi131:Power off 2023-10-06T10:37:16.489 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T10:37:16.504 INFO:teuthology.orchestra.console.smithi088:Power off 2023-10-06T10:37:16.505 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T10:37:16.514 DEBUG:teuthology.orchestra.console.smithi131:power off output: Chassis Power Control: Down/Off 2023-10-06T10:37:16.514 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:37:16.525 DEBUG:teuthology.orchestra.console.smithi088:power off output: Chassis Power Control: Down/Off 2023-10-06T10:37:16.525 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:37:16.536 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2023-10-06T10:37:16.548 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2023-10-06T10:37:20.538 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:37:20.549 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:37:20.647 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2023-10-06T10:37:20.657 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2023-10-06T10:37:24.650 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:37:24.658 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:37:24.760 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off 2023-10-06T10:37:24.766 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2023-10-06T10:37:24.861 INFO:teuthology.orchestra.console.smithi131:Power off completed 2023-10-06T10:37:24.962 INFO:teuthology.orchestra.console.smithi131:Power on 2023-10-06T10:37:24.962 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-06T10:37:24.984 DEBUG:teuthology.orchestra.console.smithi131:power on output: Chassis Power Control: Up/On 2023-10-06T10:37:24.985 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:37:25.008 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off 2023-10-06T10:37:28.770 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:37:28.880 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off 2023-10-06T10:37:28.982 INFO:teuthology.orchestra.console.smithi088:Power off completed 2023-10-06T10:37:29.009 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:37:29.083 INFO:teuthology.orchestra.console.smithi088:Power on 2023-10-06T10:37:29.083 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-06T10:37:29.105 DEBUG:teuthology.orchestra.console.smithi088:power on output: Chassis Power Control: Up/On 2023-10-06T10:37:29.105 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:37:29.122 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off 2023-10-06T10:37:29.126 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off 2023-10-06T10:37:33.126 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:37:33.135 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:37:33.236 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2023-10-06T10:37:33.246 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off 2023-10-06T10:37:33.338 INFO:teuthology.orchestra.console.smithi131:Power on completed 2023-10-06T10:37:33.438 INFO:teuthology.provision.fog.smithi131:Waiting for deploy to finish 2023-10-06T10:37:33.439 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-06T10:37:37.250 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T10:37:37.358 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2023-10-06T10:37:37.459 INFO:teuthology.orchestra.console.smithi088:Power on completed 2023-10-06T10:37:37.559 INFO:teuthology.provision.fog.smithi088:Waiting for deploy to finish 2023-10-06T10:37:37.560 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /task/active None 2023-10-06T10:37:48.502 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-06T10:37:52.618 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /task/active None 2023-10-06T10:38:03.574 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-06T10:38:07.678 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /task/active None 2023-10-06T10:38:18.638 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-06T10:38:22.734 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /task/active None 2023-10-06T10:38:33.702 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-06T10:38:37.794 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /task/active None 2023-10-06T10:38:48.774 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-06T10:38:52.856 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /task/active None 2023-10-06T10:39:03.850 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-06T10:39:07.925 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /task/active None 2023-10-06T10:39:18.926 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-06T10:39:22.994 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /task/active None 2023-10-06T10:39:34.005 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-06T10:39:38.078 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /task/active None 2023-10-06T10:39:49.082 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-06T10:39:53.154 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /task/active None 2023-10-06T10:40:04.178 DEBUG:teuthology.provision.fog.smithi131:FOG request: GET /task/active None 2023-10-06T10:40:04.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:40:08.238 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /task/active None 2023-10-06T10:40:22.654 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-10-06T10:40:23.320 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /task/active None 2023-10-06T10:40:28.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:40:31.743 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-10-06T10:40:37.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:40:38.398 DEBUG:teuthology.provision.fog.smithi088:FOG request: GET /task/active None 2023-10-06T10:40:38.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:40:40.826 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-10-06T10:40:46.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:40:49.886 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-10-06T10:40:55.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:40:56.954 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2023-10-06T10:40:58.974 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-10-06T10:41:02.958 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:41:04.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:41:06.010 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2023-10-06T10:41:08.058 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-10-06T10:41:12.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:41:14.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:41:15.066 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2023-10-06T10:41:17.114 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-10-06T10:41:21.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:41:23.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:41:24.122 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2023-10-06T10:41:26.170 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-10-06T10:41:30.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:41:32.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:41:33.178 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2023-10-06T10:41:35.226 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-10-06T10:41:39.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:41:41.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:41:42.234 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2023-10-06T10:41:44.286 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2023-10-06T10:41:48.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:41:50.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:42:48.299 WARNING:teuthology.provision.fog.smithi088:timed out 2023-10-06T10:42:50.293 WARNING:teuthology.provision.fog.smithi131:timed out 2023-10-06T10:42:54.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:42:54.536 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-06T10:42:54.536 DEBUG:teuthology.orchestra.run.smithi088:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-06T10:42:54.827 INFO:teuthology.provision.fog.smithi088:Node is ready 2023-10-06T10:42:54.827 DEBUG:teuthology.orchestra.run.smithi088:> hostname 2023-10-06T10:42:54.847 INFO:teuthology.orchestra.run.smithi088.stdout:smithi088.front.sepia.ceph.com 2023-10-06T10:42:54.848 DEBUG:teuthology.orchestra.run.smithi088:> grep smithi088.front.sepia.ceph.com /etc/hosts 2023-10-06T10:42:54.908 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88 smithi088.front.sepia.ceph.com smithi088 2023-10-06T10:42:54.909 DEBUG:teuthology.orchestra.run.smithi088:> sudo hostname smithi088 2023-10-06T10:42:55.015 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/smithi088.front.sepia.ceph.com/smithi088/g' /etc/hosts 2023-10-06T10:42:55.044 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/172.21.15.88/172.21.15.88/g' /etc/hosts 2023-10-06T10:42:55.112 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/smithi088.front.sepia.ceph.com/smithi088/g' /etc/hostname 2023-10-06T10:42:55.179 DEBUG:teuthology.orchestra.run.smithi088:> sudo hostname smithi088 2023-10-06T10:42:55.246 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2023-10-06T10:42:55.304 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Stream" 2023-10-06T10:42:55.304 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="8" 2023-10-06T10:42:55.304 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos" 2023-10-06T10:42:55.304 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora" 2023-10-06T10:42:55.304 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="8" 2023-10-06T10:42:55.304 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el8" 2023-10-06T10:42:55.304 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-06T10:42:55.304 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31" 2023-10-06T10:42:55.304 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-06T10:42:55.305 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://centos.org/" 2023-10-06T10:42:55.305 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T10:42:55.305 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T10:42:55.305 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-06T10:42:55.305 INFO:teuthology.provision.fog.smithi088:Deploy complete! 2023-10-06T10:42:55.307 DEBUG:teuthology.parallel:result is None 2023-10-06T10:42:56.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:42:56.602 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-06T10:42:56.602 DEBUG:teuthology.orchestra.run.smithi131:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-06T10:42:56.860 INFO:teuthology.provision.fog.smithi131:Node is ready 2023-10-06T10:42:56.860 DEBUG:teuthology.orchestra.run.smithi131:> hostname 2023-10-06T10:42:56.880 INFO:teuthology.orchestra.run.smithi131.stdout:smithi131.front.sepia.ceph.com 2023-10-06T10:42:56.880 DEBUG:teuthology.orchestra.run.smithi131:> grep smithi131.front.sepia.ceph.com /etc/hosts 2023-10-06T10:42:56.939 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131 smithi131.front.sepia.ceph.com smithi131 2023-10-06T10:42:56.940 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname smithi131 2023-10-06T10:42:57.046 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i -e 's/smithi131.front.sepia.ceph.com/smithi131/g' /etc/hosts 2023-10-06T10:42:57.075 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i -e 's/172.21.15.131/172.21.15.131/g' /etc/hosts 2023-10-06T10:42:57.143 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i -e 's/smithi131.front.sepia.ceph.com/smithi131/g' /etc/hostname 2023-10-06T10:42:57.210 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname smithi131 2023-10-06T10:42:57.277 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2023-10-06T10:42:57.336 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream" 2023-10-06T10:42:57.336 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="8" 2023-10-06T10:42:57.336 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos" 2023-10-06T10:42:57.336 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora" 2023-10-06T10:42:57.336 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="8" 2023-10-06T10:42:57.337 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el8" 2023-10-06T10:42:57.337 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-06T10:42:57.337 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2023-10-06T10:42:57.337 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-06T10:42:57.337 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/" 2023-10-06T10:42:57.337 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T10:42:57.337 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T10:42:57.337 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-06T10:42:57.337 INFO:teuthology.provision.fog.smithi131:Deploy complete! 2023-10-06T10:42:57.338 DEBUG:teuthology.parallel:result is None 2023-10-06T10:42:57.484 INFO:teuthology.lock.ops:Checking smithi088.front.sepia.ceph.com 2023-10-06T10:42:57.484 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-06T10:42:57.500 INFO:teuthology.lock.ops:Checking smithi131.front.sepia.ceph.com 2023-10-06T10:42:57.501 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-06T10:42:57.560 INFO:teuthology.lock.ops:Updating [smithi088.front.sepia.ceph.com]: set os type and version on server 2023-10-06T10:42:57.560 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2023-10-06T10:42:57.561 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:42:57.628 DEBUG:teuthology.orchestra.run.smithi088:> true 2023-10-06T10:42:57.699 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2023-10-06T10:42:57.699 DEBUG:teuthology.orchestra.run.smithi088:> uname -m 2023-10-06T10:42:57.758 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64 2023-10-06T10:42:57.759 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2023-10-06T10:42:57.818 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Stream" 2023-10-06T10:42:57.818 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="8" 2023-10-06T10:42:57.818 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos" 2023-10-06T10:42:57.818 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora" 2023-10-06T10:42:57.819 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="8" 2023-10-06T10:42:57.819 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el8" 2023-10-06T10:42:57.819 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-06T10:42:57.819 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31" 2023-10-06T10:42:57.819 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-06T10:42:57.819 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://centos.org/" 2023-10-06T10:42:57.819 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T10:42:57.819 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T10:42:57.819 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-06T10:42:57.820 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2023-10-06T10:42:57.851 INFO:teuthology.lock.ops:Updating [smithi131.front.sepia.ceph.com]: set os type and version on server 2023-10-06T10:42:57.851 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-10-06T10:42:57.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T10:42:57.916 DEBUG:teuthology.orchestra.run.smithi131:> true 2023-10-06T10:42:57.983 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2023-10-06T10:42:57.984 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2023-10-06T10:42:58.042 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2023-10-06T10:42:58.042 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2023-10-06T10:42:58.100 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream" 2023-10-06T10:42:58.100 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="8" 2023-10-06T10:42:58.100 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos" 2023-10-06T10:42:58.100 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora" 2023-10-06T10:42:58.100 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="8" 2023-10-06T10:42:58.100 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el8" 2023-10-06T10:42:58.100 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-06T10:42:58.100 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2023-10-06T10:42:58.101 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-06T10:42:58.101 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/" 2023-10-06T10:42:58.101 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T10:42:58.101 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T10:42:58.101 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-06T10:42:58.101 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2023-10-06T10:42:58.126 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T10:42:58.215 INFO:teuthology.dispatcher.supervisor:Running job 7412370 2023-10-06T10:42:58.215 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412370 --name yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi --description fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/dbench validater/valgrind} -- /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412370/orig.config.yaml 2023-10-06T10:42:58.218 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412370 2023-10-06T10:42:58.219 INFO:teuthology.dispatcher.supervisor:Job PID: 1093095 2023-10-06T10:42:58.219 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-10-06T10:44:58.319 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T10:46:58.491 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T10:48:58.661 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T10:50:58.827 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T10:52:58.999 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T10:54:59.163 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T10:56:59.333 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T10:58:59.503 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:00:59.667 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:02:59.825 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:04:59.951 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:07:00.115 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:09:00.275 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:11:00.439 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:13:00.607 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:15:00.759 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:17:00.925 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:19:01.091 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:21:01.263 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:23:01.431 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:25:01.597 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:27:01.763 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:29:01.931 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:31:02.095 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:33:02.276 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:35:02.455 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:37:02.616 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:39:02.772 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:39:02.973 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-10-06T11:39:03.159 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-10-06T11:39:03.159 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T11:39:03.210 DEBUG:teuthology.nuke:shortname: smithi088 2023-10-06T11:39:03.211 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T11:39:03.233 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412370', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 10:37:15.077286', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBBVU4BpyxB3/wfGg4wLELarVidN0Iko3tWZbSvsN1jQ+y7dqmlWMEx97JSxntXIZE227Nh29ZhPBW4cZfB3p28='} 2023-10-06T11:39:03.235 DEBUG:teuthology.nuke:shortname: smithi131 2023-10-06T11:39:03.235 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T11:39:03.258 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412370', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 10:37:15.079027', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHBL+SyM/p0JA3Z8qiTWDXnyGFuo2zibu+ygHR3v94/UqDa2GEPbLgP6faBb1FHr3sbcanIv0dqEG15WFLjGzUY='} 2023-10-06T11:39:03.281 INFO:teuthology.orchestra.console.smithi088:Power off 2023-10-06T11:39:03.281 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T11:39:03.302 DEBUG:teuthology.orchestra.console.smithi088:power off output: Chassis Power Control: Down/Off 2023-10-06T11:39:03.302 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T11:39:03.320 INFO:teuthology.orchestra.console.smithi131:Power off 2023-10-06T11:39:03.320 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T11:39:03.328 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off 2023-10-06T11:39:03.339 DEBUG:teuthology.orchestra.console.smithi131:power off output: Chassis Power Control: Down/Off 2023-10-06T11:39:03.340 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T11:39:03.358 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off 2023-10-06T11:39:03.429 INFO:teuthology.orchestra.console.smithi088:Power off completed 2023-10-06T11:39:03.459 INFO:teuthology.orchestra.console.smithi131:Power off completed 2023-10-06T11:39:03.580 INFO:teuthology.lock.ops:unlocked: smithi088.front.sepia.ceph.com 2023-10-06T11:39:03.605 INFO:teuthology.lock.ops:unlocked: smithi131.front.sepia.ceph.com