2023-08-15T23:05:29.353 INFO:root:teuthology version: 0.0.1.dev183+g5435569 2023-08-15T23:05:29.354 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:05:29.486 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f63ab57cee0>, signals=[15]) 2023-08-15T23:05:29.488 INFO:teuthology.lock.ops:Start node 'smithi073.front.sepia.ceph.com' reimaging 2023-08-15T23:05:29.488 INFO:teuthology.lock.ops:Updating [smithi073.front.sepia.ceph.com]: reset os type and version on server 2023-08-15T23:05:29.489 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2023-08-15T23:05:29.512 INFO:teuthology.lock.ops:Start node 'smithi179.front.sepia.ceph.com' reimaging 2023-08-15T23:05:29.512 INFO:teuthology.lock.ops:Updating [smithi179.front.sepia.ceph.com]: reset os type and version on server 2023-08-15T23:05:29.513 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2023-08-15T23:05:29.514 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /host {"name": "smithi073"} 2023-08-15T23:05:29.537 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /host {"name": "smithi179"} 2023-08-15T23:05:29.584 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-08-15T23:05:29.603 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-08-15T23:05:29.623 DEBUG:teuthology.provision.fog.smithi073:Requesting image smithi_centos_8.stream (ID 3) 2023-08-15T23:05:29.623 DEBUG:teuthology.provision.fog.smithi073:FOG request: PUT /host/178 {"imageID": 3} 2023-08-15T23:05:29.642 DEBUG:teuthology.provision.fog.smithi179:Requesting image smithi_centos_8.stream (ID 3) 2023-08-15T23:05:29.643 DEBUG:teuthology.provision.fog.smithi179:FOG request: PUT /host/284 {"imageID": 3} 2023-08-15T23:05:29.675 INFO:teuthology.provision.fog.smithi073:Scheduling deploy of centos 8.stream 2023-08-15T23:05:29.675 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None 2023-08-15T23:05:29.688 INFO:teuthology.provision.fog.smithi179:Scheduling deploy of centos 8.stream 2023-08-15T23:05:29.688 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None 2023-08-15T23:05:29.753 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /tasktype {"name": "deploy"} 2023-08-15T23:05:29.756 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /tasktype {"name": "deploy"} 2023-08-15T23:05:29.811 DEBUG:teuthology.provision.fog.smithi179:FOG request: POST /host/284/task {"taskTypeID": 1} 2023-08-15T23:05:29.814 DEBUG:teuthology.provision.fog.smithi073:FOG request: POST /host/178/task {"taskTypeID": 1} 2023-08-15T23:05:29.869 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None 2023-08-15T23:05:29.887 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None 2023-08-15T23:05:29.992 INFO:teuthology.orchestra.console.smithi073:Power off 2023-08-15T23:05:29.993 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-15T23:05:30.009 INFO:teuthology.orchestra.console.smithi179:Power off 2023-08-15T23:05:30.010 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-15T23:05:30.019 DEBUG:teuthology.orchestra.console.smithi073:power off output: Chassis Power Control: Down/Off 2023-08-15T23:05:30.020 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:05:30.031 DEBUG:teuthology.orchestra.console.smithi179:power off output: Chassis Power Control: Down/Off 2023-08-15T23:05:30.032 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:05:30.043 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is off 2023-08-15T23:05:30.054 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is off 2023-08-15T23:05:30.145 INFO:teuthology.orchestra.console.smithi073:Power off completed 2023-08-15T23:05:30.156 INFO:teuthology.orchestra.console.smithi179:Power off completed 2023-08-15T23:05:30.246 INFO:teuthology.orchestra.console.smithi073:Power on 2023-08-15T23:05:30.247 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-15T23:05:30.257 INFO:teuthology.orchestra.console.smithi179:Power on 2023-08-15T23:05:30.258 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-15T23:05:30.267 DEBUG:teuthology.orchestra.console.smithi073:power on output: Chassis Power Control: Up/On 2023-08-15T23:05:30.268 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:05:30.278 DEBUG:teuthology.orchestra.console.smithi179:power on output: Chassis Power Control: Up/On 2023-08-15T23:05:30.279 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:05:30.290 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is off 2023-08-15T23:05:30.298 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is off 2023-08-15T23:05:34.292 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:05:34.300 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:05:35.403 DEBUG:teuthology.orchestra.console.smithi073:check power output: 2023-08-15T23:05:35.410 DEBUG:teuthology.orchestra.console.smithi179:check power output: 2023-08-15T23:05:39.405 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:05:39.413 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:05:39.515 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is on 2023-08-15T23:05:39.617 INFO:teuthology.orchestra.console.smithi073:Power on completed 2023-08-15T23:05:39.717 INFO:teuthology.provision.fog.smithi073:Waiting for deploy to finish 2023-08-15T23:05:39.718 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None 2023-08-15T23:05:43.997 DEBUG:teuthology.orchestra.console.smithi179:check power output: 2023-08-15T23:05:48.000 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:05:49.109 DEBUG:teuthology.orchestra.console.smithi179:check power output: 2023-08-15T23:05:53.112 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:05:54.223 DEBUG:teuthology.orchestra.console.smithi179:check power output: 2023-08-15T23:05:58.224 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:05:59.099 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None 2023-08-15T23:05:59.334 DEBUG:teuthology.orchestra.console.smithi179:check power output: 2023-08-15T23:06:03.334 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:06:03.447 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on 2023-08-15T23:06:03.548 INFO:teuthology.orchestra.console.smithi179:Power on completed 2023-08-15T23:06:03.649 INFO:teuthology.provision.fog.smithi179:Waiting for deploy to finish 2023-08-15T23:06:03.650 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None 2023-08-15T23:06:14.196 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None 2023-08-15T23:06:19.201 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None 2023-08-15T23:06:39.414 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None 2023-08-15T23:06:39.416 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None 2023-08-15T23:06:54.492 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None 2023-08-15T23:06:59.498 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None 2023-08-15T23:07:18.597 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None 2023-08-15T23:07:18.598 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None 2023-08-15T23:07:33.679 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None 2023-08-15T23:07:33.681 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None 2023-08-15T23:07:48.760 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None 2023-08-15T23:07:48.761 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None 2023-08-15T23:08:09.320 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None 2023-08-15T23:08:09.322 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None 2023-08-15T23:08:14.393 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:08:17.456 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73 2023-08-15T23:08:23.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:08:29.384 DEBUG:teuthology.provision.fog.smithi179:FOG request: GET /task/active None 2023-08-15T23:08:34.389 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73 2023-08-15T23:08:34.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:08:44.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:08:49.456 WARNING:teuthology.provision.fog.smithi179:[Errno None] Unable to connect to port 22 on 172.21.15.179 2023-08-15T23:08:52.532 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73 2023-08-15T23:08:55.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:08:58.516 WARNING:teuthology.provision.fog.smithi179:[Errno None] Unable to connect to port 22 on 172.21.15.179 2023-08-15T23:08:58.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:09:01.584 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73 2023-08-15T23:09:04.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:09:09.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:09:14.528 WARNING:teuthology.provision.fog.smithi179:[Errno None] Unable to connect to port 22 on 172.21.15.179 2023-08-15T23:09:20.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:09:32.912 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73 2023-08-15T23:09:38.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:09:41.968 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73 2023-08-15T23:09:43.920 WARNING:teuthology.provision.fog.smithi179:[Errno None] Unable to connect to port 22 on 172.21.15.179 2023-08-15T23:09:47.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:09:48.188 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-15T23:09:48.189 DEBUG:teuthology.orchestra.run.smithi073:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-15T23:09:49.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:09:55.403 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-15T23:09:55.404 DEBUG:teuthology.orchestra.run.smithi179:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-15T23:10:08.053 INFO:teuthology.provision.fog.smithi073:Node is ready 2023-08-15T23:10:08.054 DEBUG:teuthology.orchestra.run.smithi073:> hostname 2023-08-15T23:10:08.075 INFO:teuthology.orchestra.run.smithi073.stdout:smithi193 2023-08-15T23:10:08.075 DEBUG:teuthology.orchestra.run.smithi073:> grep smithi193 /etc/hosts 2023-08-15T23:10:08.134 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.193 smithi193.front.sepia.ceph.com smithi193 2023-08-15T23:10:08.134 DEBUG:teuthology.orchestra.run.smithi073:> sudo hostname smithi073 2023-08-15T23:10:08.271 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i -e 's/smithi193/smithi073/g' /etc/hosts 2023-08-15T23:10:08.299 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i -e 's/172.21.15.193/172.21.15.73/g' /etc/hosts 2023-08-15T23:10:08.367 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i -e 's/smithi193/smithi073/g' /etc/hostname 2023-08-15T23:10:08.435 DEBUG:teuthology.orchestra.run.smithi073:> sudo hostname smithi073 2023-08-15T23:10:08.502 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release 2023-08-15T23:10:08.561 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="CentOS Stream" 2023-08-15T23:10:08.561 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="8" 2023-08-15T23:10:08.561 INFO:teuthology.orchestra.run.smithi073.stdout:ID="centos" 2023-08-15T23:10:08.561 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="rhel fedora" 2023-08-15T23:10:08.562 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="8" 2023-08-15T23:10:08.562 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el8" 2023-08-15T23:10:08.562 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-15T23:10:08.562 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31" 2023-08-15T23:10:08.562 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-15T23:10:08.562 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://centos.org/" 2023-08-15T23:10:08.562 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-15T23:10:08.563 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-15T23:10:08.563 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-15T23:10:08.563 INFO:teuthology.provision.fog.smithi073:Deploy complete! 2023-08-15T23:10:08.565 DEBUG:teuthology.parallel:result is None 2023-08-15T23:10:20.438 INFO:teuthology.provision.fog.smithi179:Node is ready 2023-08-15T23:10:20.439 DEBUG:teuthology.orchestra.run.smithi179:> hostname 2023-08-15T23:10:20.463 INFO:teuthology.orchestra.run.smithi179.stdout:smithi179.front.sepia.ceph.com 2023-08-15T23:10:20.464 DEBUG:teuthology.orchestra.run.smithi179:> grep smithi179.front.sepia.ceph.com /etc/hosts 2023-08-15T23:10:20.525 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.179 smithi179.front.sepia.ceph.com smithi179 2023-08-15T23:10:20.525 DEBUG:teuthology.orchestra.run.smithi179:> sudo hostname smithi179 2023-08-15T23:10:20.656 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i -e 's/smithi179.front.sepia.ceph.com/smithi179/g' /etc/hosts 2023-08-15T23:10:20.687 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i -e 's/172.21.15.179/172.21.15.179/g' /etc/hosts 2023-08-15T23:10:20.757 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i -e 's/smithi179.front.sepia.ceph.com/smithi179/g' /etc/hostname 2023-08-15T23:10:20.825 DEBUG:teuthology.orchestra.run.smithi179:> sudo hostname smithi179 2023-08-15T23:10:20.893 DEBUG:teuthology.orchestra.run.smithi179:> cat /etc/os-release 2023-08-15T23:10:20.950 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="CentOS Stream" 2023-08-15T23:10:20.950 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="8" 2023-08-15T23:10:20.951 INFO:teuthology.orchestra.run.smithi179.stdout:ID="centos" 2023-08-15T23:10:20.951 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="rhel fedora" 2023-08-15T23:10:20.951 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="8" 2023-08-15T23:10:20.951 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el8" 2023-08-15T23:10:20.951 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-15T23:10:20.951 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31" 2023-08-15T23:10:20.951 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-15T23:10:20.952 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://centos.org/" 2023-08-15T23:10:20.952 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-15T23:10:20.952 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-15T23:10:20.953 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-15T23:10:20.953 INFO:teuthology.provision.fog.smithi179:Deploy complete! 2023-08-15T23:10:20.954 DEBUG:teuthology.parallel:result is None 2023-08-15T23:10:26.100 INFO:teuthology.lock.ops:Checking smithi073.front.sepia.ceph.com 2023-08-15T23:10:26.101 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-15T23:10:31.119 INFO:teuthology.lock.ops:Checking smithi179.front.sepia.ceph.com 2023-08-15T23:10:31.119 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-15T23:10:36.145 INFO:teuthology.lock.ops:Updating [smithi073.front.sepia.ceph.com]: set os type and version on server 2023-08-15T23:10:36.145 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2023-08-15T23:10:36.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:10:36.214 DEBUG:teuthology.orchestra.run.smithi073:> true 2023-08-15T23:10:36.308 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2023-08-15T23:10:36.309 DEBUG:teuthology.orchestra.run.smithi073:> uname -m 2023-08-15T23:10:36.366 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64 2023-08-15T23:10:36.367 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release 2023-08-15T23:10:36.425 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="CentOS Stream" 2023-08-15T23:10:36.426 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="8" 2023-08-15T23:10:36.426 INFO:teuthology.orchestra.run.smithi073.stdout:ID="centos" 2023-08-15T23:10:36.426 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="rhel fedora" 2023-08-15T23:10:36.426 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="8" 2023-08-15T23:10:36.426 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el8" 2023-08-15T23:10:36.427 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-15T23:10:36.427 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31" 2023-08-15T23:10:36.427 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-15T23:10:36.427 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://centos.org/" 2023-08-15T23:10:36.427 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-15T23:10:36.427 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-15T23:10:36.427 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-15T23:10:36.428 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2023-08-15T23:10:40.876 INFO:teuthology.lock.ops:Updating [smithi179.front.sepia.ceph.com]: set os type and version on server 2023-08-15T23:10:40.876 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2023-08-15T23:10:40.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:10:40.943 DEBUG:teuthology.orchestra.run.smithi179:> true 2023-08-15T23:10:41.084 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2023-08-15T23:10:41.084 DEBUG:teuthology.orchestra.run.smithi179:> uname -m 2023-08-15T23:10:41.143 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2023-08-15T23:10:41.144 DEBUG:teuthology.orchestra.run.smithi179:> cat /etc/os-release 2023-08-15T23:10:41.204 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="CentOS Stream" 2023-08-15T23:10:41.205 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="8" 2023-08-15T23:10:41.205 INFO:teuthology.orchestra.run.smithi179.stdout:ID="centos" 2023-08-15T23:10:41.205 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="rhel fedora" 2023-08-15T23:10:41.205 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="8" 2023-08-15T23:10:41.205 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el8" 2023-08-15T23:10:41.205 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-15T23:10:41.206 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31" 2023-08-15T23:10:41.206 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-15T23:10:41.206 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://centos.org/" 2023-08-15T23:10:41.206 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-15T23:10:41.206 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-15T23:10:41.206 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-15T23:10:41.207 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2023-08-15T23:10:41.810 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:10:41.911 INFO:teuthology.dispatcher.supervisor:Running job 7368982 2023-08-15T23:10:41.912 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-08-15_16:56:50-fs-wip-yuri7-testing-2023-08-15-0741-pacific-distro-default-smithi/7368982 --name yuriw-2023-08-15_16:56:50-fs-wip-yuri7-testing-2023-08-15-0741-pacific-distro-default-smithi --description fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} -- /home/teuthworker/archive/yuriw-2023-08-15_16:56:50-fs-wip-yuri7-testing-2023-08-15-0741-pacific-distro-default-smithi/7368982/orig.config.yaml 2023-08-15T23:10:41.917 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-08-15_16:56:50-fs-wip-yuri7-testing-2023-08-15-0741-pacific-distro-default-smithi/7368982 2023-08-15T23:10:41.918 INFO:teuthology.dispatcher.supervisor:Job PID: 7719 2023-08-15T23:10:41.919 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-08-15T23:12:42.016 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:14:47.180 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:16:52.361 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:18:57.541 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:21:00.255 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:23:05.429 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:25:10.609 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:27:15.776 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:29:19.276 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:31:19.869 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:33:25.041 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:35:30.188 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:37:35.349 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:39:40.517 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:41:45.693 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:41:49.488 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-08-15T23:41:49.599 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-08-15T23:41:49.600 INFO:teuthology.nuke:Checking targets against current locks 2023-08-15T23:41:49.638 DEBUG:teuthology.nuke:shortname: smithi073 2023-08-15T23:41:49.639 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-15T23:41:49.656 DEBUG:teuthology.nuke:shortname: smithi179 2023-08-15T23:41:49.656 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-15T23:41:49.659 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-08-15_16:56:50-fs-wip-yuri7-testing-2023-08-15-0741-pacific-distro-default-smithi/7368982', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-15 23:05:18.102093', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7+KGt3nzz1D36sz8t67nwS6Us7srYDUdbQMLZ7EaUpRilp1dYS7DuX55aZuaO1r5r4BSRrvOJgfF2Y3IHQbtnKyYpYQeCMvcPtKUGsUX5WimDLV04eGUTZRgqnX4yntZ6AHtrwep/+3ihn65WuDr6zKPVYoncvNAFogVNjs8Z+3b1bc4Dl3Es9eLj9TqJYSjR4I3r4zDLMdSw+D/tq3o4rl5KbDh+uyp4rCcznF5Pj03+N6fEbKsUjkc7gaHAwbVVZlF36QM7+MoJLHTpoGkssToSHsG4xDvg+/QkSIZmB6Zu19mlI+TpWYkS01CAHKMkG4G+VkK1hRvNrXE+Tizqh2MuyMZmdC9Nq0ullNXgPWW+JXziefQuBiEJblBcBnp5LEhwzpqEuX73k8ZEFYSUJ1wXkuuGwfH28kJUYwhXDCVqXEEVlhpAdoKZcq+6L1mtTfkZdftFQ2nToP+9x/bg7BNRrCLx9o3nSRbPURNJj9AMGATMDYeN6g5/cYNtwG8='} 2023-08-15T23:41:49.680 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-08-15_16:56:50-fs-wip-yuri7-testing-2023-08-15-0741-pacific-distro-default-smithi/7368982', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-15 23:05:18.104095', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIFxKv23ejvdwhXVJvSYuTnhIyq382a4GNN9gs/wWUOg6aVCzxNllGH8G//teU1oBB48/ouFz/EUY20DWA/XBfdeYkNmZIflQipLurC89rzn3p2CYEHGQxf3wQMGI95ZqiS6SKy83t3C/ReKHgHtSkLNg/AOhnZiYi2Iy2ZuR5VK7qrclPcUpzuqbgeJ77eIeUgvgyO3E6XdXXW2xD3Os4OqKroaoWqsyBZZxfTaTrNjEz2JlYC+Y0yApLtcmUzysvxTqFBjDg8MMwvgx2v3UOSo1miqHMNnPXsBtbbtKxDdcm7DN9YG0j1KJeYN3h7JA4R1O218qpSf6R9XR9U0dDsV9/rwXYNPUKhDUbf2sAzi307xWV1Q15cb4c+9C1NjALbu7v2EwmBEbFGlCwtXFvcPKNMTd7vyXz41ZhqHUJJnZUb5UNndVw4GqmQtUO5Eb7e4a4vo8Cq3MmnEEZ7N7EI7D5KagRtHort36AyGHsfFoXzkWEYDeRaGMu9IjA5JU='} 2023-08-15T23:41:49.703 INFO:teuthology.orchestra.console.smithi073:Power off 2023-08-15T23:41:49.704 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-15T23:41:49.723 DEBUG:teuthology.orchestra.console.smithi073:power off output: Chassis Power Control: Down/Off 2023-08-15T23:41:49.724 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:41:49.740 INFO:teuthology.orchestra.console.smithi179:Power off 2023-08-15T23:41:49.741 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-15T23:41:49.750 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is off 2023-08-15T23:41:49.851 INFO:teuthology.orchestra.console.smithi073:Power off completed 2023-08-15T23:41:59.993 INFO:teuthology.lock.ops:unlocked: smithi073.front.sepia.ceph.com 2023-08-15T23:42:04.770 DEBUG:teuthology.orchestra.console.smithi179:power off output: Chassis Power Control: Down/Off 2023-08-15T23:42:04.770 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:42:05.780 DEBUG:teuthology.orchestra.console.smithi179:check power output: 2023-08-15T23:42:09.784 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:42:10.894 DEBUG:teuthology.orchestra.console.smithi179:check power output: 2023-08-15T23:42:14.896 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:42:16.006 DEBUG:teuthology.orchestra.console.smithi179:check power output: 2023-08-15T23:42:20.008 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:42:21.118 DEBUG:teuthology.orchestra.console.smithi179:check power output: 2023-08-15T23:42:25.120 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:42:26.230 DEBUG:teuthology.orchestra.console.smithi179:check power output: 2023-08-15T23:42:30.232 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:42:31.341 DEBUG:teuthology.orchestra.console.smithi179:check power output: 2023-08-15T23:42:35.343 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:42:35.453 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is off 2023-08-15T23:42:35.554 INFO:teuthology.orchestra.console.smithi179:Power off completed 2023-08-15T23:42:35.888 INFO:teuthology.lock.ops:unlocked: smithi179.front.sepia.ceph.com