2023-10-21T19:28:56.027 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-21T19:28:56.046 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T19:28:56.183 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fce9b95fee0>, signals=[15]) 2023-10-21T19:28:56.185 INFO:teuthology.lock.ops:Start node 'smithi098.front.sepia.ceph.com' reimaging 2023-10-21T19:28:56.185 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: reset os type and version on server 2023-10-21T19:28:56.185 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2023-10-21T19:28:56.204 INFO:teuthology.lock.ops:Start node 'smithi192.front.sepia.ceph.com' reimaging 2023-10-21T19:28:56.204 INFO:teuthology.lock.ops:Updating [smithi192.front.sepia.ceph.com]: reset os type and version on server 2023-10-21T19:28:56.205 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2023-10-21T19:28:56.206 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /host {"name": "smithi098"} 2023-10-21T19:28:56.229 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /host {"name": "smithi192"} 2023-10-21T19:28:56.296 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-10-21T19:28:56.306 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-10-21T19:28:56.342 DEBUG:teuthology.provision.fog.smithi098:Requesting image smithi_centos_8.stream (ID 3) 2023-10-21T19:28:56.342 DEBUG:teuthology.provision.fog.smithi098:FOG request: PUT /host/203 {"imageID": 3} 2023-10-21T19:28:56.350 DEBUG:teuthology.provision.fog.smithi192:Requesting image smithi_centos_8.stream (ID 3) 2023-10-21T19:28:56.350 DEBUG:teuthology.provision.fog.smithi192:FOG request: PUT /host/297 {"imageID": 3} 2023-10-21T19:28:56.394 INFO:teuthology.provision.fog.smithi192:Scheduling deploy of centos 8.stream 2023-10-21T19:28:56.394 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None 2023-10-21T19:28:56.397 INFO:teuthology.provision.fog.smithi098:Scheduling deploy of centos 8.stream 2023-10-21T19:28:56.397 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-10-21T19:28:56.442 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /tasktype {"name": "deploy"} 2023-10-21T19:28:56.449 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /tasktype {"name": "deploy"} 2023-10-21T19:28:56.485 DEBUG:teuthology.provision.fog.smithi192:FOG request: POST /host/297/task {"taskTypeID": 1} 2023-10-21T19:28:56.491 DEBUG:teuthology.provision.fog.smithi098:FOG request: POST /host/203/task {"taskTypeID": 1} 2023-10-21T19:28:56.536 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None 2023-10-21T19:28:56.539 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-10-21T19:28:56.620 INFO:teuthology.orchestra.console.smithi098:Power off 2023-10-21T19:28:56.621 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-21T19:28:56.633 INFO:teuthology.orchestra.console.smithi192:Power off 2023-10-21T19:28:56.634 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-21T19:28:56.641 DEBUG:teuthology.orchestra.console.smithi098:power off output: Chassis Power Control: Down/Off 2023-10-21T19:28:56.641 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T19:28:56.655 DEBUG:teuthology.orchestra.console.smithi192:power off output: Chassis Power Control: Down/Off 2023-10-21T19:28:56.656 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T19:28:56.664 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off 2023-10-21T19:28:56.677 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is off 2023-10-21T19:28:56.765 INFO:teuthology.orchestra.console.smithi098:Power off completed 2023-10-21T19:28:56.778 INFO:teuthology.orchestra.console.smithi192:Power off completed 2023-10-21T19:28:56.866 INFO:teuthology.orchestra.console.smithi098:Power on 2023-10-21T19:28:56.866 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-21T19:28:56.880 INFO:teuthology.orchestra.console.smithi192:Power on 2023-10-21T19:28:56.880 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-21T19:28:56.894 DEBUG:teuthology.orchestra.console.smithi098:power on output: Chassis Power Control: Up/On 2023-10-21T19:28:56.894 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T19:28:56.903 DEBUG:teuthology.orchestra.console.smithi192:power on output: Chassis Power Control: Up/On 2023-10-21T19:28:56.903 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T19:28:56.916 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off 2023-10-21T19:28:56.923 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is off 2023-10-21T19:29:00.918 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T19:29:00.925 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T19:29:01.027 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off 2023-10-21T19:29:01.033 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is off 2023-10-21T19:29:05.032 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T19:29:05.039 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T19:29:05.141 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on 2023-10-21T19:29:05.149 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2023-10-21T19:29:05.243 INFO:teuthology.orchestra.console.smithi098:Power on completed 2023-10-21T19:29:05.250 INFO:teuthology.orchestra.console.smithi192:Power on completed 2023-10-21T19:29:05.344 INFO:teuthology.provision.fog.smithi098:Waiting for deploy to finish 2023-10-21T19:29:05.344 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-10-21T19:29:05.351 INFO:teuthology.provision.fog.smithi192:Waiting for deploy to finish 2023-10-21T19:29:05.351 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None 2023-10-21T19:29:20.408 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-10-21T19:29:20.409 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None 2023-10-21T19:29:35.484 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None 2023-10-21T19:29:35.485 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-10-21T19:29:50.559 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None 2023-10-21T19:29:50.561 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-10-21T19:30:05.627 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-10-21T19:30:05.629 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None 2023-10-21T19:30:20.705 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-10-21T19:30:20.707 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None 2023-10-21T19:30:35.788 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None 2023-10-21T19:30:35.790 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-10-21T19:30:50.860 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None 2023-10-21T19:30:50.861 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-10-21T19:31:05.972 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None 2023-10-21T19:31:05.973 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-10-21T19:31:21.062 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-10-21T19:31:21.064 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None 2023-10-21T19:31:36.150 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-10-21T19:31:36.152 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None 2023-10-21T19:31:36.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T19:31:51.237 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None 2023-10-21T19:31:51.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T19:31:54.618 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-10-21T19:32:00.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T19:32:03.706 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-10-21T19:32:09.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T19:32:09.722 WARNING:teuthology.provision.fog.smithi192:[Errno None] Unable to connect to port 22 on 172.21.15.192 2023-10-21T19:32:12.794 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-10-21T19:32:15.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T19:32:18.782 WARNING:teuthology.provision.fog.smithi192:[Errno None] Unable to connect to port 22 on 172.21.15.192 2023-10-21T19:32:18.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T19:32:21.854 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-10-21T19:32:24.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T19:32:27.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T19:32:27.866 WARNING:teuthology.provision.fog.smithi192:[Errno None] Unable to connect to port 22 on 172.21.15.192 2023-10-21T19:32:30.942 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-10-21T19:32:33.874 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T19:32:36.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T19:32:36.996 WARNING:teuthology.provision.fog.smithi192:[Errno None] Unable to connect to port 22 on 172.21.15.192 2023-10-21T19:32:43.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T19:33:37.050 WARNING:teuthology.provision.fog.smithi098:timed out 2023-10-21T19:33:43.010 WARNING:teuthology.provision.fog.smithi192:timed out 2023-10-21T19:33:43.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T19:33:43.372 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-21T19:33:43.373 DEBUG:teuthology.orchestra.run.smithi098:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-21T19:33:43.610 INFO:teuthology.provision.fog.smithi098:Node is ready 2023-10-21T19:33:43.610 DEBUG:teuthology.orchestra.run.smithi098:> hostname 2023-10-21T19:33:43.669 INFO:teuthology.orchestra.run.smithi098.stdout:smithi098.front.sepia.ceph.com 2023-10-21T19:33:43.670 DEBUG:teuthology.orchestra.run.smithi098:> grep smithi098.front.sepia.ceph.com /etc/hosts 2023-10-21T19:33:43.729 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98 smithi098.front.sepia.ceph.com smithi098 2023-10-21T19:33:43.730 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname smithi098 2023-10-21T19:33:43.839 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/smithi098.front.sepia.ceph.com/smithi098/g' /etc/hosts 2023-10-21T19:33:43.869 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/172.21.15.98/172.21.15.98/g' /etc/hosts 2023-10-21T19:33:43.938 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/smithi098.front.sepia.ceph.com/smithi098/g' /etc/hostname 2023-10-21T19:33:44.007 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname smithi098 2023-10-21T19:33:44.073 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2023-10-21T19:33:44.131 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream" 2023-10-21T19:33:44.132 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8" 2023-10-21T19:33:44.132 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2023-10-21T19:33:44.132 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2023-10-21T19:33:44.132 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8" 2023-10-21T19:33:44.133 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8" 2023-10-21T19:33:44.133 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-21T19:33:44.133 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2023-10-21T19:33:44.133 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-21T19:33:44.134 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/" 2023-10-21T19:33:44.134 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-21T19:33:44.134 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-21T19:33:44.134 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-21T19:33:44.135 INFO:teuthology.provision.fog.smithi098:Deploy complete! 2023-10-21T19:33:44.136 DEBUG:teuthology.parallel:result is None 2023-10-21T19:33:49.014 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T19:33:49.253 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-21T19:33:49.254 DEBUG:teuthology.orchestra.run.smithi192:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-21T19:33:49.489 INFO:teuthology.provision.fog.smithi192:Node is ready 2023-10-21T19:33:49.489 DEBUG:teuthology.orchestra.run.smithi192:> hostname 2023-10-21T19:33:49.508 INFO:teuthology.orchestra.run.smithi192.stdout:smithi192.front.sepia.ceph.com 2023-10-21T19:33:49.508 DEBUG:teuthology.orchestra.run.smithi192:> grep smithi192.front.sepia.ceph.com /etc/hosts 2023-10-21T19:33:49.568 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.15.192 smithi192.front.sepia.ceph.com smithi192 2023-10-21T19:33:49.569 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname smithi192 2023-10-21T19:33:49.669 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i -e 's/smithi192.front.sepia.ceph.com/smithi192/g' /etc/hosts 2023-10-21T19:33:49.699 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i -e 's/172.21.15.192/172.21.15.192/g' /etc/hosts 2023-10-21T19:33:49.768 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i -e 's/smithi192.front.sepia.ceph.com/smithi192/g' /etc/hostname 2023-10-21T19:33:49.836 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname smithi192 2023-10-21T19:33:49.903 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2023-10-21T19:33:49.962 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2023-10-21T19:33:49.962 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="8" 2023-10-21T19:33:49.963 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2023-10-21T19:33:49.963 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2023-10-21T19:33:49.963 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="8" 2023-10-21T19:33:49.964 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el8" 2023-10-21T19:33:49.964 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-21T19:33:49.964 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2023-10-21T19:33:49.964 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-21T19:33:49.964 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2023-10-21T19:33:49.965 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-21T19:33:49.965 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-21T19:33:49.965 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-21T19:33:49.965 INFO:teuthology.provision.fog.smithi192:Deploy complete! 2023-10-21T19:33:49.966 DEBUG:teuthology.parallel:result is None 2023-10-21T19:33:50.149 INFO:teuthology.lock.ops:Checking smithi098.front.sepia.ceph.com 2023-10-21T19:33:50.150 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-21T19:33:50.178 INFO:teuthology.lock.ops:Checking smithi192.front.sepia.ceph.com 2023-10-21T19:33:50.178 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-21T19:33:50.204 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: set os type and version on server 2023-10-21T19:33:50.205 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-10-21T19:33:50.205 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T19:33:50.272 DEBUG:teuthology.orchestra.run.smithi098:> true 2023-10-21T19:33:50.342 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-10-21T19:33:50.343 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2023-10-21T19:33:50.403 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2023-10-21T19:33:50.403 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2023-10-21T19:33:50.462 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream" 2023-10-21T19:33:50.462 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8" 2023-10-21T19:33:50.462 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2023-10-21T19:33:50.462 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2023-10-21T19:33:50.463 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8" 2023-10-21T19:33:50.463 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8" 2023-10-21T19:33:50.463 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-21T19:33:50.463 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2023-10-21T19:33:50.463 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-21T19:33:50.464 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/" 2023-10-21T19:33:50.464 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-21T19:33:50.464 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-21T19:33:50.464 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-21T19:33:50.465 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2023-10-21T19:33:50.486 INFO:teuthology.lock.ops:Updating [smithi192.front.sepia.ceph.com]: set os type and version on server 2023-10-21T19:33:50.486 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-10-21T19:33:50.487 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T19:33:50.552 DEBUG:teuthology.orchestra.run.smithi192:> true 2023-10-21T19:33:50.619 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-10-21T19:33:50.620 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2023-10-21T19:33:50.677 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2023-10-21T19:33:50.678 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2023-10-21T19:33:50.735 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2023-10-21T19:33:50.735 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="8" 2023-10-21T19:33:50.736 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2023-10-21T19:33:50.736 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2023-10-21T19:33:50.736 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="8" 2023-10-21T19:33:50.736 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el8" 2023-10-21T19:33:50.737 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-21T19:33:50.737 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2023-10-21T19:33:50.737 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-21T19:33:50.737 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2023-10-21T19:33:50.737 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-21T19:33:50.738 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-21T19:33:50.738 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-21T19:33:50.738 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2023-10-21T19:33:50.761 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T19:33:50.849 INFO:teuthology.dispatcher.supervisor:Running job 7427764 2023-10-21T19:33:50.849 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2023-10-14_14:23:11-upgrade:quincy-x-reef-distro-default-smithi/7427764 --name teuthology-2023-10-14_14:23:11-upgrade:quincy-x-reef-distro-default-smithi --description upgrade:quincy-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} -- /home/teuthworker/archive/teuthology-2023-10-14_14:23:11-upgrade:quincy-x-reef-distro-default-smithi/7427764/orig.config.yaml 2023-10-21T19:33:50.853 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2023-10-14_14:23:11-upgrade:quincy-x-reef-distro-default-smithi/7427764 2023-10-21T19:33:50.854 INFO:teuthology.dispatcher.supervisor:Job PID: 602639 2023-10-21T19:33:50.854 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-10-21T19:35:50.954 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T19:37:51.131 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T19:39:51.287 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T19:41:51.451 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T19:43:51.619 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T19:45:51.815 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T19:47:51.983 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T19:49:52.151 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T19:51:52.323 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T19:53:52.487 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T19:55:52.652 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T19:57:52.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T19:59:52.983 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:01:53.115 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:03:53.285 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:05:53.447 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:07:53.614 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:09:53.767 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:11:53.944 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:13:54.114 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:15:54.275 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:17:54.444 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:19:54.607 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:21:54.771 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:23:54.938 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:25:55.106 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:27:55.303 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:29:55.471 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:31:55.623 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:33:55.787 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:35:55.943 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:37:56.117 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:39:56.283 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:41:56.448 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:43:56.623 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:45:56.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:47:56.995 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:49:57.167 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T20:49:57.239 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-10-21T20:49:57.427 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-10-21T20:49:57.428 INFO:teuthology.nuke:Checking targets against current locks 2023-10-21T20:49:57.465 DEBUG:teuthology.nuke:shortname: smithi098 2023-10-21T20:49:57.466 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-21T20:49:57.492 DEBUG:teuthology.nuke:shortname: smithi192 2023-10-21T20:49:57.493 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-21T20:49:57.496 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2023-10-14_14:23:11-upgrade:quincy-x-reef-distro-default-smithi/7427764', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-21 19:28:50.037981', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHfYiEFtvY0Sq7h5ijHHzhZN2wwr7usZEl3hUVpJ9PJClVeTFOQivAJ569V1aMoPkbC1x2vqEVhy1wFlmfGIYMo='} 2023-10-21T20:49:57.510 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2023-10-14_14:23:11-upgrade:quincy-x-reef-distro-default-smithi/7427764', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-21 19:28:50.036708', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKySpSHStUdoxM8btpOGiDc8st/48W1lvhIWDsMobOO0tbQ6km9omz8gLSMZwNYejvZBV+51FfF4+w96wtGFYL8='} 2023-10-21T20:49:57.538 INFO:teuthology.orchestra.console.smithi098:Power off 2023-10-21T20:49:57.538 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-21T20:49:57.560 DEBUG:teuthology.orchestra.console.smithi098:power off output: Chassis Power Control: Down/Off 2023-10-21T20:49:57.560 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T20:49:57.573 INFO:teuthology.orchestra.console.smithi192:Power off 2023-10-21T20:49:57.574 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-21T20:49:57.582 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off 2023-10-21T20:49:57.596 DEBUG:teuthology.orchestra.console.smithi192:power off output: Chassis Power Control: Down/Off 2023-10-21T20:49:57.597 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T20:49:57.618 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is off 2023-10-21T20:49:57.684 INFO:teuthology.orchestra.console.smithi098:Power off completed 2023-10-21T20:49:57.719 INFO:teuthology.orchestra.console.smithi192:Power off completed 2023-10-21T20:49:57.839 INFO:teuthology.lock.ops:unlocked: smithi098.front.sepia.ceph.com 2023-10-21T20:49:57.864 INFO:teuthology.lock.ops:unlocked: smithi192.front.sepia.ceph.com