2023-11-13T22:25:49.158 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-13T22:25:49.160 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T22:25:49.296 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2373af2670>, signals=[15]) 2023-11-13T22:25:49.297 INFO:teuthology.lock.ops:Start node 'smithi098.front.sepia.ceph.com' reimaging 2023-11-13T22:25:49.297 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: reset os type and version on server 2023-11-13T22:25:49.298 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2023-11-13T22:25:49.316 INFO:teuthology.lock.ops:Start node 'smithi173.front.sepia.ceph.com' reimaging 2023-11-13T22:25:49.316 INFO:teuthology.lock.ops:Updating [smithi173.front.sepia.ceph.com]: reset os type and version on server 2023-11-13T22:25:49.316 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2023-11-13T22:25:49.317 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /host {"name": "smithi098"} 2023-11-13T22:25:49.341 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /host {"name": "smithi173"} 2023-11-13T22:25:49.393 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-11-13T22:25:49.405 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-11-13T22:25:49.435 DEBUG:teuthology.provision.fog.smithi098:Requesting image smithi_centos_8.stream (ID 3) 2023-11-13T22:25:49.435 DEBUG:teuthology.provision.fog.smithi098:FOG request: PUT /host/203 {"imageID": 3} 2023-11-13T22:25:49.444 DEBUG:teuthology.provision.fog.smithi173:Requesting image smithi_centos_8.stream (ID 3) 2023-11-13T22:25:49.444 DEBUG:teuthology.provision.fog.smithi173:FOG request: PUT /host/278 {"imageID": 3} 2023-11-13T22:25:49.478 INFO:teuthology.provision.fog.smithi098:Scheduling deploy of centos 8.stream 2023-11-13T22:25:49.479 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-11-13T22:25:49.483 INFO:teuthology.provision.fog.smithi173:Scheduling deploy of centos 8.stream 2023-11-13T22:25:49.484 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-11-13T22:25:49.708 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /tasktype {"name": "deploy"} 2023-11-13T22:25:49.720 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /tasktype {"name": "deploy"} 2023-11-13T22:25:49.746 DEBUG:teuthology.provision.fog.smithi098:FOG request: POST /host/203/task {"taskTypeID": 1} 2023-11-13T22:25:49.763 DEBUG:teuthology.provision.fog.smithi173:FOG request: POST /host/278/task {"taskTypeID": 1} 2023-11-13T22:25:49.799 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-11-13T22:25:49.814 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-11-13T22:25:50.054 INFO:teuthology.orchestra.console.smithi173:Power off 2023-11-13T22:25:50.055 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-13T22:25:50.069 INFO:teuthology.orchestra.console.smithi098:Power off 2023-11-13T22:25:50.070 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-13T22:25:50.078 DEBUG:teuthology.orchestra.console.smithi173:power off output: Chassis Power Control: Down/Off 2023-11-13T22:25:50.079 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T22:25:50.094 DEBUG:teuthology.orchestra.console.smithi098:power off output: Chassis Power Control: Down/Off 2023-11-13T22:25:50.095 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T22:25:50.102 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2023-11-13T22:25:50.118 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on 2023-11-13T22:25:50.203 INFO:teuthology.orchestra.console.smithi173:Power off completed 2023-11-13T22:25:50.304 INFO:teuthology.orchestra.console.smithi173:Power on 2023-11-13T22:25:50.304 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-13T22:25:50.326 DEBUG:teuthology.orchestra.console.smithi173:power on output: Chassis Power Control: Up/On 2023-11-13T22:25:50.326 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T22:25:50.350 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2023-11-13T22:25:54.122 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T22:25:54.231 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on 2023-11-13T22:25:54.351 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T22:25:54.460 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2023-11-13T22:25:58.236 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T22:25:58.345 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on 2023-11-13T22:25:58.461 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T22:25:58.569 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2023-11-13T22:25:58.670 INFO:teuthology.orchestra.console.smithi173:Power on completed 2023-11-13T22:25:58.771 INFO:teuthology.provision.fog.smithi173:Waiting for deploy to finish 2023-11-13T22:25:58.772 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-11-13T22:26:02.350 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T22:26:02.459 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off 2023-11-13T22:26:02.560 INFO:teuthology.orchestra.console.smithi098:Power off completed 2023-11-13T22:26:02.661 INFO:teuthology.orchestra.console.smithi098:Power on 2023-11-13T22:26:02.661 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-13T22:26:02.682 DEBUG:teuthology.orchestra.console.smithi098:power on output: Chassis Power Control: Up/On 2023-11-13T22:26:02.682 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T22:26:02.702 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off 2023-11-13T22:26:06.706 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T22:26:06.816 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off 2023-11-13T22:26:10.819 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T22:26:10.927 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on 2023-11-13T22:26:11.028 INFO:teuthology.orchestra.console.smithi098:Power on completed 2023-11-13T22:26:11.130 INFO:teuthology.provision.fog.smithi098:Waiting for deploy to finish 2023-11-13T22:26:11.131 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-11-13T22:26:14.028 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-11-13T22:26:27.065 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-11-13T22:26:30.153 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-11-13T22:26:42.914 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-11-13T22:26:45.653 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-11-13T22:27:00.483 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-11-13T22:27:02.399 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-11-13T22:27:20.326 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-11-13T22:27:21.071 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-11-13T22:27:45.115 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-11-13T22:27:45.187 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-11-13T22:28:12.754 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-11-13T22:28:13.364 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-11-13T22:28:45.634 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-11-13T22:28:49.786 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-11-13T22:29:03.624 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-11-13T22:29:13.443 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-11-13T22:29:23.887 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-11-13T22:29:33.039 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T22:29:35.370 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-11-13T22:29:50.952 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-11-13T22:29:51.358 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2023-11-13T22:29:54.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T22:29:57.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T22:30:00.443 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2023-11-13T22:30:06.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T22:30:13.115 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-11-13T22:30:19.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T22:30:22.202 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-11-13T22:30:28.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T22:30:31.290 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-11-13T22:30:37.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T22:30:40.346 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-11-13T22:30:46.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T22:30:49.435 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-11-13T22:30:55.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T22:31:06.463 WARNING:teuthology.provision.fog.smithi173:timed out 2023-11-13T22:31:12.471 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T22:31:12.724 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-13T22:31:12.725 DEBUG:teuthology.orchestra.run.smithi173:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-13T22:31:12.985 INFO:teuthology.provision.fog.smithi173:Node is ready 2023-11-13T22:31:12.985 DEBUG:teuthology.orchestra.run.smithi173:> hostname 2023-11-13T22:31:13.005 INFO:teuthology.orchestra.run.smithi173.stdout:smithi173.front.sepia.ceph.com 2023-11-13T22:31:13.005 DEBUG:teuthology.orchestra.run.smithi173:> grep smithi173.front.sepia.ceph.com /etc/hosts 2023-11-13T22:31:13.064 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.15.173 smithi173.front.sepia.ceph.com smithi173 2023-11-13T22:31:13.064 DEBUG:teuthology.orchestra.run.smithi173:> sudo hostname smithi173 2023-11-13T22:31:13.163 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/smithi173.front.sepia.ceph.com/smithi173/g' /etc/hosts 2023-11-13T22:31:13.193 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/172.21.15.173/172.21.15.173/g' /etc/hosts 2023-11-13T22:31:13.262 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/smithi173.front.sepia.ceph.com/smithi173/g' /etc/hostname 2023-11-13T22:31:13.331 DEBUG:teuthology.orchestra.run.smithi173:> sudo hostname smithi173 2023-11-13T22:31:13.399 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2023-11-13T22:31:13.457 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2023-11-13T22:31:13.457 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="8" 2023-11-13T22:31:13.458 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2023-11-13T22:31:13.458 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2023-11-13T22:31:13.458 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="8" 2023-11-13T22:31:13.459 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el8" 2023-11-13T22:31:13.459 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-13T22:31:13.459 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2023-11-13T22:31:13.460 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-13T22:31:13.460 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2023-11-13T22:31:13.460 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-13T22:31:13.460 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-13T22:31:13.461 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-13T22:31:13.462 INFO:teuthology.provision.fog.smithi173:Deploy complete! 2023-11-13T22:31:13.464 DEBUG:teuthology.parallel:result is None 2023-11-13T22:31:13.786 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-11-13T22:31:19.793 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T22:31:22.874 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-11-13T22:31:28.882 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T22:31:32.235 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-13T22:31:32.236 DEBUG:teuthology.orchestra.run.smithi098:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-13T22:31:57.978 INFO:teuthology.provision.fog.smithi098:Node is ready 2023-11-13T22:31:57.978 DEBUG:teuthology.orchestra.run.smithi098:> hostname 2023-11-13T22:31:58.003 INFO:teuthology.orchestra.run.smithi098.stdout:smithi098.front.sepia.ceph.com 2023-11-13T22:31:58.004 DEBUG:teuthology.orchestra.run.smithi098:> grep smithi098.front.sepia.ceph.com /etc/hosts 2023-11-13T22:31:58.066 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98 smithi098.front.sepia.ceph.com smithi098 2023-11-13T22:31:58.066 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname smithi098 2023-11-13T22:31:58.183 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/smithi098.front.sepia.ceph.com/smithi098/g' /etc/hosts 2023-11-13T22:31:58.217 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/172.21.15.98/172.21.15.98/g' /etc/hosts 2023-11-13T22:31:58.288 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/smithi098.front.sepia.ceph.com/smithi098/g' /etc/hostname 2023-11-13T22:31:58.358 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname smithi098 2023-11-13T22:31:58.427 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2023-11-13T22:31:58.486 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream" 2023-11-13T22:31:58.487 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8" 2023-11-13T22:31:58.487 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2023-11-13T22:31:58.487 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2023-11-13T22:31:58.487 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8" 2023-11-13T22:31:58.488 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8" 2023-11-13T22:31:58.488 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-13T22:31:58.488 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2023-11-13T22:31:58.488 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-13T22:31:58.489 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/" 2023-11-13T22:31:58.489 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-13T22:31:58.489 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-13T22:31:58.490 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-13T22:31:58.490 INFO:teuthology.provision.fog.smithi098:Deploy complete! 2023-11-13T22:31:58.490 DEBUG:teuthology.parallel:result is None 2023-11-13T22:31:58.705 INFO:teuthology.lock.ops:Checking smithi098.front.sepia.ceph.com 2023-11-13T22:31:58.705 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-13T22:31:58.732 INFO:teuthology.lock.ops:Checking smithi173.front.sepia.ceph.com 2023-11-13T22:31:58.732 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-13T22:31:58.750 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: set os type and version on server 2023-11-13T22:31:58.750 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-11-13T22:31:58.751 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T22:31:58.816 DEBUG:teuthology.orchestra.run.smithi098:> true 2023-11-13T22:31:58.888 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-11-13T22:31:58.889 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2023-11-13T22:31:58.947 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2023-11-13T22:31:58.947 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2023-11-13T22:31:59.005 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream" 2023-11-13T22:31:59.006 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8" 2023-11-13T22:31:59.006 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2023-11-13T22:31:59.006 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2023-11-13T22:31:59.007 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8" 2023-11-13T22:31:59.007 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8" 2023-11-13T22:31:59.007 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-13T22:31:59.007 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2023-11-13T22:31:59.008 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-13T22:31:59.008 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/" 2023-11-13T22:31:59.008 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-13T22:31:59.009 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-13T22:31:59.009 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-13T22:31:59.010 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2023-11-13T22:31:59.032 INFO:teuthology.lock.ops:Updating [smithi173.front.sepia.ceph.com]: set os type and version on server 2023-11-13T22:31:59.033 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2023-11-13T22:31:59.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-13T22:31:59.104 DEBUG:teuthology.orchestra.run.smithi173:> true 2023-11-13T22:31:59.242 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2023-11-13T22:31:59.243 DEBUG:teuthology.orchestra.run.smithi173:> uname -m 2023-11-13T22:31:59.302 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2023-11-13T22:31:59.302 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2023-11-13T22:31:59.362 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2023-11-13T22:31:59.362 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="8" 2023-11-13T22:31:59.362 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2023-11-13T22:31:59.363 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2023-11-13T22:31:59.363 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="8" 2023-11-13T22:31:59.363 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el8" 2023-11-13T22:31:59.364 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-13T22:31:59.364 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2023-11-13T22:31:59.364 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-13T22:31:59.365 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2023-11-13T22:31:59.365 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-13T22:31:59.365 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-13T22:31:59.366 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-13T22:31:59.366 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2023-11-13T22:31:59.391 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T22:31:59.469 INFO:teuthology.dispatcher.supervisor:Running job 7456500 2023-11-13T22:31:59.470 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-11-13_22:25:01-orch-reef-release-distro-default-smithi/7456500 --name yuriw-2023-11-13_22:25:01-orch-reef-release-distro-default-smithi --description orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} -- /home/teuthworker/archive/yuriw-2023-11-13_22:25:01-orch-reef-release-distro-default-smithi/7456500/orig.config.yaml 2023-11-13T22:31:59.474 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-11-13_22:25:01-orch-reef-release-distro-default-smithi/7456500 2023-11-13T22:31:59.475 INFO:teuthology.dispatcher.supervisor:Job PID: 2314199 2023-11-13T22:31:59.475 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-13T22:33:59.576 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T22:35:59.743 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T22:37:59.911 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T22:40:00.078 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T22:42:00.250 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T22:44:00.441 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T22:46:00.616 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T22:48:00.787 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T22:50:00.928 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T22:52:01.035 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T22:54:01.203 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T22:56:01.359 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T22:58:01.511 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:00:01.603 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:02:01.751 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:04:01.915 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:06:02.092 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:08:02.277 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:10:02.447 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:12:02.616 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:14:02.783 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:16:02.948 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:18:03.122 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:20:03.295 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:22:03.479 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:24:03.639 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:26:03.799 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:28:03.975 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:30:04.139 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:32:04.299 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:34:04.467 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:36:04.636 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:38:04.812 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:40:05.043 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:42:05.211 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:44:05.386 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:46:05.567 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:48:05.737 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:50:05.904 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:52:06.067 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:54:06.235 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-13T23:54:06.305 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-11-13T23:54:06.409 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-13T23:54:06.409 INFO:teuthology.nuke:Checking targets against current locks 2023-11-13T23:54:06.458 DEBUG:teuthology.nuke:shortname: smithi098 2023-11-13T23:54:06.459 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-13T23:54:06.485 DEBUG:teuthology.nuke:shortname: smithi173 2023-11-13T23:54:06.486 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-13T23:54:06.489 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-13_22:25:01-orch-reef-release-distro-default-smithi/7456500', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-13 22:25:48.417927', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDpVtQYP0yUGjyiolHNeY0+BvFR2R8tYjgwXJ3n6HDwChZUSDQDiDzNfVJSLZUxMaXOGwIabmtX1j3mESyLoZbU='} 2023-11-13T23:54:06.513 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-13_22:25:01-orch-reef-release-distro-default-smithi/7456500', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-13 22:25:48.415989', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB1aTxewhhs+dpX+QZazPBPMZeLfQgmtLuvpUN8NA0x0xgMBz8Bx/nX4cHAJbbqMO5zoO7oQoilcL1G9KEGrJbI='} 2023-11-13T23:54:06.542 INFO:teuthology.orchestra.console.smithi098:Power off 2023-11-13T23:54:06.543 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-13T23:54:06.557 INFO:teuthology.orchestra.console.smithi173:Power off 2023-11-13T23:54:06.557 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-13T23:54:06.567 DEBUG:teuthology.orchestra.console.smithi098:power off output: Chassis Power Control: Down/Off 2023-11-13T23:54:06.568 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T23:54:06.579 DEBUG:teuthology.orchestra.console.smithi173:power off output: Chassis Power Control: Down/Off 2023-11-13T23:54:06.580 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-13T23:54:06.595 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off 2023-11-13T23:54:06.608 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2023-11-13T23:54:06.697 INFO:teuthology.orchestra.console.smithi098:Power off completed 2023-11-13T23:54:06.709 INFO:teuthology.orchestra.console.smithi173:Power off completed 2023-11-13T23:54:06.853 INFO:teuthology.lock.ops:unlocked: smithi098.front.sepia.ceph.com 2023-11-13T23:54:06.862 INFO:teuthology.lock.ops:unlocked: smithi173.front.sepia.ceph.com