2023-09-16T14:24:32.848 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-16T14:24:32.849 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T14:24:33.092 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f49b0794ee0>, signals=[15]) 2023-09-16T14:24:33.093 INFO:teuthology.lock.ops:Start node 'smithi003.front.sepia.ceph.com' reimaging 2023-09-16T14:24:33.094 INFO:teuthology.lock.ops:Updating [smithi003.front.sepia.ceph.com]: reset os type and version on server 2023-09-16T14:24:33.094 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2023-09-16T14:24:33.114 INFO:teuthology.lock.ops:Start node 'smithi183.front.sepia.ceph.com' reimaging 2023-09-16T14:24:33.115 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: reset os type and version on server 2023-09-16T14:24:33.115 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2023-09-16T14:24:33.116 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /host {"name": "smithi003"} 2023-09-16T14:24:33.139 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /host {"name": "smithi183"} 2023-09-16T14:24:33.190 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-09-16T14:24:33.210 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-09-16T14:24:33.232 DEBUG:teuthology.provision.fog.smithi003:Requesting image smithi_centos_8.stream (ID 3) 2023-09-16T14:24:33.232 DEBUG:teuthology.provision.fog.smithi003:FOG request: PUT /host/109 {"imageID": 3} 2023-09-16T14:24:33.256 DEBUG:teuthology.provision.fog.smithi183:Requesting image smithi_centos_8.stream (ID 3) 2023-09-16T14:24:33.256 DEBUG:teuthology.provision.fog.smithi183:FOG request: PUT /host/288 {"imageID": 3} 2023-09-16T14:24:33.272 INFO:teuthology.provision.fog.smithi003:Scheduling deploy of centos 8.stream 2023-09-16T14:24:33.272 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-09-16T14:24:33.301 INFO:teuthology.provision.fog.smithi183:Scheduling deploy of centos 8.stream 2023-09-16T14:24:33.301 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:24:33.356 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /tasktype {"name": "deploy"} 2023-09-16T14:24:33.383 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /tasktype {"name": "deploy"} 2023-09-16T14:24:33.400 DEBUG:teuthology.provision.fog.smithi003:FOG request: POST /host/109/task {"taskTypeID": 1} 2023-09-16T14:24:33.414 DEBUG:teuthology.provision.fog.smithi183:FOG request: POST /host/288/task {"taskTypeID": 1} 2023-09-16T14:24:33.462 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-09-16T14:24:33.465 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:24:33.594 INFO:teuthology.orchestra.console.smithi003:Power off 2023-09-16T14:24:33.595 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-16T14:24:33.605 INFO:teuthology.orchestra.console.smithi183:Power off 2023-09-16T14:24:33.605 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-16T14:24:33.614 DEBUG:teuthology.orchestra.console.smithi003:power off output: Chassis Power Control: Down/Off 2023-09-16T14:24:33.615 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-16T14:24:33.625 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off 2023-09-16T14:24:33.626 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-16T14:24:33.634 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2023-09-16T14:24:33.645 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-09-16T14:24:37.638 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-16T14:24:37.646 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-16T14:24:37.746 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2023-09-16T14:24:37.755 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-09-16T14:24:41.750 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-16T14:24:41.756 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-16T14:24:41.858 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off 2023-09-16T14:24:41.864 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-09-16T14:24:41.960 INFO:teuthology.orchestra.console.smithi003:Power off completed 2023-09-16T14:24:42.061 INFO:teuthology.orchestra.console.smithi003:Power on 2023-09-16T14:24:42.061 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-16T14:24:42.081 DEBUG:teuthology.orchestra.console.smithi003:power on output: Chassis Power Control: Up/On 2023-09-16T14:24:42.082 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-16T14:24:42.101 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off 2023-09-16T14:24:45.866 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-16T14:24:45.974 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2023-09-16T14:24:46.075 INFO:teuthology.orchestra.console.smithi183:Power off completed 2023-09-16T14:24:46.103 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-16T14:24:46.177 INFO:teuthology.orchestra.console.smithi183:Power on 2023-09-16T14:24:46.177 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-16T14:24:46.198 DEBUG:teuthology.orchestra.console.smithi183:power on output: Chassis Power Control: Up/On 2023-09-16T14:24:46.198 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-16T14:24:46.213 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off 2023-09-16T14:24:46.218 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2023-09-16T14:24:50.218 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-16T14:24:50.224 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-16T14:24:50.326 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2023-09-16T14:24:50.331 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2023-09-16T14:24:50.427 INFO:teuthology.orchestra.console.smithi003:Power on completed 2023-09-16T14:24:50.528 INFO:teuthology.provision.fog.smithi003:Waiting for deploy to finish 2023-09-16T14:24:50.529 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-09-16T14:24:54.336 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-16T14:24:54.562 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-09-16T14:24:54.663 INFO:teuthology.orchestra.console.smithi183:Power on completed 2023-09-16T14:24:54.764 INFO:teuthology.provision.fog.smithi183:Waiting for deploy to finish 2023-09-16T14:24:54.764 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:25:05.770 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-09-16T14:25:09.975 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:25:20.986 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-09-16T14:25:25.185 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:25:36.182 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-09-16T14:25:40.384 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:25:51.391 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-09-16T14:25:55.598 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:26:06.594 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-09-16T14:26:10.857 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:26:21.870 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-09-16T14:26:26.141 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:26:38.270 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-09-16T14:26:41.400 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:26:54.782 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-09-16T14:26:56.705 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:27:10.481 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-09-16T14:27:12.442 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:27:25.783 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-09-16T14:27:28.945 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:27:41.098 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-09-16T14:27:44.717 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:27:56.396 DEBUG:teuthology.provision.fog.smithi003:FOG request: GET /task/active None 2023-09-16T14:27:56.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:27:59.866 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-09-16T14:27:59.947 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:28:05.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:28:08.954 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-09-16T14:28:14.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:28:15.427 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:28:18.042 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-09-16T14:28:24.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:28:27.130 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-09-16T14:28:30.834 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:28:33.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:28:36.187 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3 2023-09-16T14:28:42.190 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:28:46.261 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:29:01.706 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:29:17.140 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:29:32.477 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:29:42.202 WARNING:teuthology.provision.fog.smithi003:timed out 2023-09-16T14:29:47.742 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:29:48.204 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:29:48.468 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-16T14:29:48.469 DEBUG:teuthology.orchestra.run.smithi003:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-16T14:29:48.728 INFO:teuthology.provision.fog.smithi003:Node is ready 2023-09-16T14:29:48.729 DEBUG:teuthology.orchestra.run.smithi003:> hostname 2023-09-16T14:29:48.748 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003.front.sepia.ceph.com 2023-09-16T14:29:48.749 DEBUG:teuthology.orchestra.run.smithi003:> grep smithi003.front.sepia.ceph.com /etc/hosts 2023-09-16T14:29:48.808 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3 smithi003.front.sepia.ceph.com smithi003 2023-09-16T14:29:48.809 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname smithi003 2023-09-16T14:29:48.918 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/smithi003.front.sepia.ceph.com/smithi003/g' /etc/hosts 2023-09-16T14:29:48.949 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/172.21.15.3/172.21.15.3/g' /etc/hosts 2023-09-16T14:29:49.018 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/smithi003.front.sepia.ceph.com/smithi003/g' /etc/hostname 2023-09-16T14:29:49.087 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname smithi003 2023-09-16T14:29:49.155 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2023-09-16T14:29:49.213 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="CentOS Stream" 2023-09-16T14:29:49.213 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="8" 2023-09-16T14:29:49.214 INFO:teuthology.orchestra.run.smithi003.stdout:ID="centos" 2023-09-16T14:29:49.214 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="rhel fedora" 2023-09-16T14:29:49.214 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="8" 2023-09-16T14:29:49.214 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el8" 2023-09-16T14:29:49.214 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-16T14:29:49.215 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31" 2023-09-16T14:29:49.215 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-16T14:29:49.215 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://centos.org/" 2023-09-16T14:29:49.215 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-16T14:29:49.215 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-16T14:29:49.216 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-16T14:29:49.216 INFO:teuthology.provision.fog.smithi003:Deploy complete! 2023-09-16T14:29:49.217 DEBUG:teuthology.parallel:result is None 2023-09-16T14:30:03.070 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:30:18.412 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:30:33.698 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:30:48.982 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:31:04.190 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:31:19.358 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:31:34.518 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:31:49.666 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:32:04.814 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:32:19.919 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:32:35.022 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:32:50.104 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-09-16T14:32:50.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:32:53.242 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-09-16T14:32:59.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:33:02.302 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-09-16T14:33:08.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:33:11.386 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-09-16T14:33:17.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:33:20.442 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-09-16T14:33:26.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:33:29.498 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-09-16T14:33:35.502 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:33:53.818 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-09-16T14:33:59.825 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:34:02.906 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-09-16T14:34:08.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:34:11.962 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-09-16T14:34:17.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:34:18.194 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-16T14:34:18.194 DEBUG:teuthology.orchestra.run.smithi183:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-16T14:34:38.133 INFO:teuthology.provision.fog.smithi183:Node is ready 2023-09-16T14:34:38.133 DEBUG:teuthology.orchestra.run.smithi183:> hostname 2023-09-16T14:34:38.158 INFO:teuthology.orchestra.run.smithi183.stdout:smithi183.front.sepia.ceph.com 2023-09-16T14:34:38.161 DEBUG:teuthology.orchestra.run.smithi183:> grep smithi183.front.sepia.ceph.com /etc/hosts 2023-09-16T14:34:38.218 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.15.183 smithi183.front.sepia.ceph.com smithi183 2023-09-16T14:34:38.219 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname smithi183 2023-09-16T14:34:38.342 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/smithi183.front.sepia.ceph.com/smithi183/g' /etc/hosts 2023-09-16T14:34:38.374 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/172.21.15.183/172.21.15.183/g' /etc/hosts 2023-09-16T14:34:38.445 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/smithi183.front.sepia.ceph.com/smithi183/g' /etc/hostname 2023-09-16T14:34:38.515 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname smithi183 2023-09-16T14:34:38.583 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2023-09-16T14:34:38.641 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2023-09-16T14:34:38.642 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="8" 2023-09-16T14:34:38.642 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2023-09-16T14:34:38.642 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2023-09-16T14:34:38.642 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="8" 2023-09-16T14:34:38.643 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el8" 2023-09-16T14:34:38.643 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-16T14:34:38.643 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2023-09-16T14:34:38.643 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-16T14:34:38.644 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2023-09-16T14:34:38.644 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-16T14:34:38.644 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-16T14:34:38.644 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-16T14:34:38.645 INFO:teuthology.provision.fog.smithi183:Deploy complete! 2023-09-16T14:34:38.645 DEBUG:teuthology.parallel:result is None 2023-09-16T14:34:38.806 INFO:teuthology.lock.ops:Checking smithi003.front.sepia.ceph.com 2023-09-16T14:34:38.807 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-16T14:34:38.841 INFO:teuthology.lock.ops:Checking smithi183.front.sepia.ceph.com 2023-09-16T14:34:38.842 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-16T14:34:38.867 INFO:teuthology.lock.ops:Updating [smithi003.front.sepia.ceph.com]: set os type and version on server 2023-09-16T14:34:38.867 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-09-16T14:34:38.868 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:34:38.938 DEBUG:teuthology.orchestra.run.smithi003:> true 2023-09-16T14:34:39.082 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2023-09-16T14:34:39.082 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2023-09-16T14:34:39.141 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2023-09-16T14:34:39.142 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2023-09-16T14:34:39.204 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="CentOS Stream" 2023-09-16T14:34:39.204 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="8" 2023-09-16T14:34:39.205 INFO:teuthology.orchestra.run.smithi003.stdout:ID="centos" 2023-09-16T14:34:39.205 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="rhel fedora" 2023-09-16T14:34:39.205 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="8" 2023-09-16T14:34:39.205 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el8" 2023-09-16T14:34:39.206 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-16T14:34:39.206 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31" 2023-09-16T14:34:39.206 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-16T14:34:39.206 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://centos.org/" 2023-09-16T14:34:39.206 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-16T14:34:39.207 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-16T14:34:39.207 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-16T14:34:39.207 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2023-09-16T14:34:39.224 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: set os type and version on server 2023-09-16T14:34:39.225 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-09-16T14:34:39.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-16T14:34:39.292 DEBUG:teuthology.orchestra.run.smithi183:> true 2023-09-16T14:34:39.361 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-09-16T14:34:39.362 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2023-09-16T14:34:39.421 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2023-09-16T14:34:39.421 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2023-09-16T14:34:39.479 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2023-09-16T14:34:39.480 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="8" 2023-09-16T14:34:39.480 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2023-09-16T14:34:39.480 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2023-09-16T14:34:39.480 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="8" 2023-09-16T14:34:39.481 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el8" 2023-09-16T14:34:39.481 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-16T14:34:39.481 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2023-09-16T14:34:39.481 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-16T14:34:39.481 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2023-09-16T14:34:39.482 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-16T14:34:39.482 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-16T14:34:39.482 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-16T14:34:39.483 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2023-09-16T14:34:39.508 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T14:34:39.625 INFO:teuthology.dispatcher.supervisor:Running job 7397215 2023-09-16T14:34:39.626 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2023-09-16_14:23:09-upgrade:pacific-x-reef-distro-default-smithi/7397215 --name teuthology-2023-09-16_14:23:09-upgrade:pacific-x-reef-distro-default-smithi --description upgrade:pacific-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/connectivity} -- /home/teuthworker/archive/teuthology-2023-09-16_14:23:09-upgrade:pacific-x-reef-distro-default-smithi/7397215/orig.config.yaml 2023-09-16T14:34:39.629 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2023-09-16_14:23:09-upgrade:pacific-x-reef-distro-default-smithi/7397215 2023-09-16T14:34:39.630 INFO:teuthology.dispatcher.supervisor:Job PID: 3483571 2023-09-16T14:34:39.630 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-09-16T14:36:39.730 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T14:38:39.923 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T14:40:40.106 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T14:42:40.269 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T14:44:40.447 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T14:46:40.612 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T14:48:40.764 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T14:50:40.917 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T14:52:41.079 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T14:54:41.239 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T14:56:41.390 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T14:58:41.546 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:00:41.697 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:02:41.847 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:04:41.904 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:06:42.049 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:08:42.214 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:10:42.330 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:12:42.491 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:14:42.650 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:16:42.817 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:18:42.991 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:20:43.154 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:22:43.270 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:24:43.427 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:26:43.582 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:28:43.718 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:30:43.879 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:32:44.038 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:34:44.196 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:36:44.360 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:38:44.530 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:40:44.692 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:42:44.859 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:44:45.023 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:46:45.177 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:48:45.345 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:50:45.472 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:52:45.562 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:54:45.757 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:56:45.933 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T15:58:46.131 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T16:00:46.293 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T16:02:46.451 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T16:04:46.623 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T16:06:46.795 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T16:08:47.128 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-16T16:08:47.204 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-09-16T16:08:47.407 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-09-16T16:08:47.408 INFO:teuthology.nuke:Checking targets against current locks 2023-09-16T16:08:47.455 DEBUG:teuthology.nuke:shortname: smithi003 2023-09-16T16:08:47.456 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-16T16:08:47.481 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2023-09-16_14:23:09-upgrade:pacific-x-reef-distro-default-smithi/7397215', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-16 14:24:32.046989', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEvc0z3uDbAHbq8JNELedPdDPm/q6I91DJJk/X30iDc37POQ3zY8zvvdC/9B5Z8ToZ2GQpH+JSngWWz+abDgEgg='} 2023-09-16T16:08:47.484 DEBUG:teuthology.nuke:shortname: smithi183 2023-09-16T16:08:47.484 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-16T16:08:47.505 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2023-09-16_14:23:09-upgrade:pacific-x-reef-distro-default-smithi/7397215', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-16 14:24:32.045316', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP5WoRqCrfLdSE66jhjOg6rwO4AkkPjCj0ZT59k6kI7ZjnmsAQNiFaomvp9zFSgZVvnlc3MpuWhV1uNgJdB0d8s='} 2023-09-16T16:08:47.537 INFO:teuthology.orchestra.console.smithi003:Power off 2023-09-16T16:08:47.537 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-16T16:08:47.560 DEBUG:teuthology.orchestra.console.smithi003:power off output: Chassis Power Control: Down/Off 2023-09-16T16:08:47.561 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-16T16:08:47.573 INFO:teuthology.orchestra.console.smithi183:Power off 2023-09-16T16:08:47.574 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-16T16:08:47.583 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off 2023-09-16T16:08:47.594 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off 2023-09-16T16:08:47.595 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-16T16:08:47.619 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2023-09-16T16:08:47.685 INFO:teuthology.orchestra.console.smithi003:Power off completed 2023-09-16T16:08:47.720 INFO:teuthology.orchestra.console.smithi183:Power off completed 2023-09-16T16:08:47.836 INFO:teuthology.lock.ops:unlocked: smithi003.front.sepia.ceph.com 2023-09-16T16:08:47.861 INFO:teuthology.lock.ops:unlocked: smithi183.front.sepia.ceph.com