2023-08-11T13:27:28.634 INFO:root:teuthology version: 0.0.1.dev181+g7fda959 2023-08-11T13:27:28.636 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:27:28.782 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f06256a7f70>, signals=[15]) 2023-08-11T13:27:28.785 INFO:teuthology.lock.ops:Start node 'smithi118.front.sepia.ceph.com' reimaging 2023-08-11T13:27:28.785 INFO:teuthology.lock.ops:Updating [smithi118.front.sepia.ceph.com]: reset os type and version on server 2023-08-11T13:27:28.786 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2023-08-11T13:27:28.808 INFO:teuthology.lock.ops:Start node 'smithi183.front.sepia.ceph.com' reimaging 2023-08-11T13:27:28.809 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: reset os type and version on server 2023-08-11T13:27:28.810 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2023-08-11T13:27:28.811 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /host {"name": "smithi118"} 2023-08-11T13:27:28.832 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /host {"name": "smithi183"} 2023-08-11T13:27:28.885 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-08-11T13:27:28.922 DEBUG:teuthology.provision.fog.smithi118:Requesting image smithi_centos_8.stream (ID 3) 2023-08-11T13:27:28.922 DEBUG:teuthology.provision.fog.smithi118:FOG request: PUT /host/223 {"imageID": 3} 2023-08-11T13:27:28.925 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-08-11T13:27:28.961 DEBUG:teuthology.provision.fog.smithi183:Requesting image smithi_centos_8.stream (ID 3) 2023-08-11T13:27:28.961 DEBUG:teuthology.provision.fog.smithi183:FOG request: PUT /host/288 {"imageID": 3} 2023-08-11T13:27:28.976 INFO:teuthology.provision.fog.smithi118:Scheduling deploy of centos 8.stream 2023-08-11T13:27:28.976 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-08-11T13:27:29.006 INFO:teuthology.provision.fog.smithi183:Scheduling deploy of centos 8.stream 2023-08-11T13:27:29.007 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-08-11T13:27:29.027 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /tasktype {"name": "deploy"} 2023-08-11T13:27:29.052 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /tasktype {"name": "deploy"} 2023-08-11T13:27:29.064 DEBUG:teuthology.provision.fog.smithi118:FOG request: POST /host/223/task {"taskTypeID": 1} 2023-08-11T13:27:29.086 DEBUG:teuthology.provision.fog.smithi183:FOG request: POST /host/288/task {"taskTypeID": 1} 2023-08-11T13:27:29.130 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-08-11T13:27:29.147 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-08-11T13:27:29.222 INFO:teuthology.orchestra.console.smithi118:Power off 2023-08-11T13:27:29.223 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-11T13:27:29.243 INFO:teuthology.orchestra.console.smithi183:Power off 2023-08-11T13:27:29.244 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-11T13:27:29.255 DEBUG:teuthology.orchestra.console.smithi118:power off output: Chassis Power Control: Down/Off 2023-08-11T13:27:29.256 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T13:27:29.277 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off 2023-08-11T13:27:29.278 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T13:27:29.294 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2023-08-11T13:27:29.309 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-08-11T13:27:33.296 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T13:27:33.310 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T13:27:33.409 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2023-08-11T13:27:33.420 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-08-11T13:27:37.413 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T13:27:37.423 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T13:27:37.526 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2023-08-11T13:27:37.534 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2023-08-11T13:27:37.628 INFO:teuthology.orchestra.console.smithi118:Power off completed 2023-08-11T13:27:37.635 INFO:teuthology.orchestra.console.smithi183:Power off completed 2023-08-11T13:27:37.729 INFO:teuthology.orchestra.console.smithi118:Power on 2023-08-11T13:27:37.729 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-11T13:27:37.738 INFO:teuthology.orchestra.console.smithi183:Power on 2023-08-11T13:27:37.739 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-11T13:27:37.753 DEBUG:teuthology.orchestra.console.smithi118:power on output: Chassis Power Control: Up/On 2023-08-11T13:27:37.754 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T13:27:37.766 DEBUG:teuthology.orchestra.console.smithi183:power on output: Chassis Power Control: Up/On 2023-08-11T13:27:37.767 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T13:27:37.776 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2023-08-11T13:27:37.789 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2023-08-11T13:27:41.780 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T13:27:41.790 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T13:27:41.892 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2023-08-11T13:27:41.904 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2023-08-11T13:27:45.896 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T13:27:45.905 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T13:27:46.054 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-08-11T13:27:46.108 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2023-08-11T13:27:46.156 INFO:teuthology.orchestra.console.smithi183:Power on completed 2023-08-11T13:27:46.209 INFO:teuthology.orchestra.console.smithi118:Power on completed 2023-08-11T13:27:46.257 INFO:teuthology.provision.fog.smithi183:Waiting for deploy to finish 2023-08-11T13:27:46.258 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-08-11T13:27:46.311 INFO:teuthology.provision.fog.smithi118:Waiting for deploy to finish 2023-08-11T13:27:46.311 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-08-11T13:28:01.328 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-08-11T13:28:01.365 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-08-11T13:28:16.396 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-08-11T13:28:16.414 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-08-11T13:28:31.477 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-08-11T13:28:31.479 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-08-11T13:28:46.564 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-08-11T13:28:46.566 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-08-11T13:29:01.643 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-08-11T13:29:01.645 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-08-11T13:29:16.737 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-08-11T13:29:16.738 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-08-11T13:29:31.816 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-08-11T13:29:31.817 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-08-11T13:29:46.892 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-08-11T13:29:46.894 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-08-11T13:30:01.968 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-08-11T13:30:01.969 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-08-11T13:30:17.044 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-08-11T13:30:17.045 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-08-11T13:30:32.124 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-08-11T13:30:32.125 DEBUG:teuthology.provision.fog.smithi183:FOG request: GET /task/active None 2023-08-11T13:30:32.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:30:35.248 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-08-11T13:30:41.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:30:44.304 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-08-11T13:30:47.188 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-08-11T13:30:47.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:30:50.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:30:50.320 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-08-11T13:30:53.360 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-08-11T13:30:56.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:30:59.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:30:59.376 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-08-11T13:31:02.416 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-08-11T13:31:05.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:31:08.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:31:08.433 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-08-11T13:31:11.472 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-08-11T13:31:14.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:31:17.476 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:31:17.488 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-08-11T13:31:20.528 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-08-11T13:31:23.493 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:31:26.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:31:26.544 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-08-11T13:31:32.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:31:35.600 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-08-11T13:31:41.604 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:31:44.656 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-08-11T13:31:50.660 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:31:53.712 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-08-11T13:31:59.716 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:32:26.560 WARNING:teuthology.provision.fog.smithi183:timed out 2023-08-11T13:32:32.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:32:32.807 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-11T13:32:32.809 DEBUG:teuthology.orchestra.run.smithi183:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-11T13:32:33.041 INFO:teuthology.provision.fog.smithi183:Node is ready 2023-08-11T13:32:33.041 DEBUG:teuthology.orchestra.run.smithi183:> hostname 2023-08-11T13:32:33.101 INFO:teuthology.orchestra.run.smithi183.stdout:smithi183.front.sepia.ceph.com 2023-08-11T13:32:33.101 DEBUG:teuthology.orchestra.run.smithi183:> grep smithi183.front.sepia.ceph.com /etc/hosts 2023-08-11T13:32:33.162 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.15.183 smithi183.front.sepia.ceph.com smithi183 2023-08-11T13:32:33.162 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname smithi183 2023-08-11T13:32:33.263 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/smithi183.front.sepia.ceph.com/smithi183/g' /etc/hosts 2023-08-11T13:32:33.294 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/172.21.15.183/172.21.15.183/g' /etc/hosts 2023-08-11T13:32:33.363 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/smithi183.front.sepia.ceph.com/smithi183/g' /etc/hostname 2023-08-11T13:32:33.432 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname smithi183 2023-08-11T13:32:33.500 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2023-08-11T13:32:33.558 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2023-08-11T13:32:33.559 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="8" 2023-08-11T13:32:33.559 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2023-08-11T13:32:33.559 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2023-08-11T13:32:33.560 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="8" 2023-08-11T13:32:33.561 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el8" 2023-08-11T13:32:33.561 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-11T13:32:33.562 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2023-08-11T13:32:33.562 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-11T13:32:33.563 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2023-08-11T13:32:33.566 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-11T13:32:33.566 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-11T13:32:33.566 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-11T13:32:33.567 INFO:teuthology.provision.fog.smithi183:Deploy complete! 2023-08-11T13:32:33.568 DEBUG:teuthology.parallel:result is None 2023-08-11T13:32:59.744 WARNING:teuthology.provision.fog.smithi118:timed out 2023-08-11T13:33:05.748 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:33:05.951 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-11T13:33:05.952 DEBUG:teuthology.orchestra.run.smithi118:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-11T13:33:06.204 INFO:teuthology.provision.fog.smithi118:Node is ready 2023-08-11T13:33:06.205 DEBUG:teuthology.orchestra.run.smithi118:> hostname 2023-08-11T13:33:06.226 INFO:teuthology.orchestra.run.smithi118.stdout:smithi118.front.sepia.ceph.com 2023-08-11T13:33:06.227 DEBUG:teuthology.orchestra.run.smithi118:> grep smithi118.front.sepia.ceph.com /etc/hosts 2023-08-11T13:33:06.288 INFO:teuthology.orchestra.run.smithi118.stdout:172.21.15.118 smithi118.front.sepia.ceph.com smithi118 2023-08-11T13:33:06.288 DEBUG:teuthology.orchestra.run.smithi118:> sudo hostname smithi118 2023-08-11T13:33:06.399 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/smithi118.front.sepia.ceph.com/smithi118/g' /etc/hosts 2023-08-11T13:33:06.434 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/172.21.15.118/172.21.15.118/g' /etc/hosts 2023-08-11T13:33:06.509 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/smithi118.front.sepia.ceph.com/smithi118/g' /etc/hostname 2023-08-11T13:33:06.580 DEBUG:teuthology.orchestra.run.smithi118:> sudo hostname smithi118 2023-08-11T13:33:06.651 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2023-08-11T13:33:06.711 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="CentOS Stream" 2023-08-11T13:33:06.711 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="8" 2023-08-11T13:33:06.712 INFO:teuthology.orchestra.run.smithi118.stdout:ID="centos" 2023-08-11T13:33:06.712 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE="rhel fedora" 2023-08-11T13:33:06.712 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="8" 2023-08-11T13:33:06.712 INFO:teuthology.orchestra.run.smithi118.stdout:PLATFORM_ID="platform:el8" 2023-08-11T13:33:06.712 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-11T13:33:06.713 INFO:teuthology.orchestra.run.smithi118.stdout:ANSI_COLOR="0;31" 2023-08-11T13:33:06.713 INFO:teuthology.orchestra.run.smithi118.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-11T13:33:06.713 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://centos.org/" 2023-08-11T13:33:06.713 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-11T13:33:06.713 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-11T13:33:06.714 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-11T13:33:06.714 INFO:teuthology.provision.fog.smithi118:Deploy complete! 2023-08-11T13:33:06.714 DEBUG:teuthology.parallel:result is None 2023-08-11T13:33:06.893 INFO:teuthology.lock.ops:Checking smithi118.front.sepia.ceph.com 2023-08-11T13:33:06.894 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-11T13:33:06.914 INFO:teuthology.lock.ops:Checking smithi183.front.sepia.ceph.com 2023-08-11T13:33:06.915 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-11T13:33:06.932 INFO:teuthology.lock.ops:Updating [smithi118.front.sepia.ceph.com]: set os type and version on server 2023-08-11T13:33:06.933 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2023-08-11T13:33:06.934 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:33:07.001 DEBUG:teuthology.orchestra.run.smithi118:> true 2023-08-11T13:33:07.072 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2023-08-11T13:33:07.073 DEBUG:teuthology.orchestra.run.smithi118:> uname -m 2023-08-11T13:33:07.132 INFO:teuthology.orchestra.run.smithi118.stdout:x86_64 2023-08-11T13:33:07.133 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2023-08-11T13:33:07.192 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="CentOS Stream" 2023-08-11T13:33:07.192 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="8" 2023-08-11T13:33:07.193 INFO:teuthology.orchestra.run.smithi118.stdout:ID="centos" 2023-08-11T13:33:07.193 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE="rhel fedora" 2023-08-11T13:33:07.193 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="8" 2023-08-11T13:33:07.193 INFO:teuthology.orchestra.run.smithi118.stdout:PLATFORM_ID="platform:el8" 2023-08-11T13:33:07.193 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-11T13:33:07.194 INFO:teuthology.orchestra.run.smithi118.stdout:ANSI_COLOR="0;31" 2023-08-11T13:33:07.194 INFO:teuthology.orchestra.run.smithi118.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-11T13:33:07.194 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://centos.org/" 2023-08-11T13:33:07.194 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-11T13:33:07.195 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-11T13:33:07.196 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-11T13:33:07.197 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2023-08-11T13:33:07.223 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: set os type and version on server 2023-08-11T13:33:07.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-08-11T13:33:07.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T13:33:07.334 DEBUG:teuthology.orchestra.run.smithi183:> true 2023-08-11T13:33:07.472 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-08-11T13:33:07.473 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2023-08-11T13:33:07.532 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2023-08-11T13:33:07.532 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2023-08-11T13:33:07.591 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2023-08-11T13:33:07.591 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="8" 2023-08-11T13:33:07.591 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2023-08-11T13:33:07.591 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2023-08-11T13:33:07.591 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="8" 2023-08-11T13:33:07.592 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el8" 2023-08-11T13:33:07.592 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-11T13:33:07.592 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2023-08-11T13:33:07.592 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-11T13:33:07.592 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2023-08-11T13:33:07.592 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-11T13:33:07.593 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-11T13:33:07.593 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-11T13:33:07.593 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2023-08-11T13:33:07.614 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:33:07.714 INFO:teuthology.dispatcher.supervisor:Running job 7365916 2023-08-11T13:33:07.714 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365916 --name rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365916/orig.config.yaml 2023-08-11T13:33:07.718 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365916 2023-08-11T13:33:07.718 INFO:teuthology.dispatcher.supervisor:Job PID: 5589 2023-08-11T13:33:07.719 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-08-11T13:35:07.816 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:37:07.972 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:39:08.134 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:41:08.301 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:43:08.653 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:45:08.833 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:47:09.025 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:49:09.189 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:51:09.362 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:53:09.533 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:55:09.701 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:57:09.869 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:59:10.037 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:59:10.108 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-08-11T13:59:10.299 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-08-11T13:59:10.300 INFO:teuthology.nuke:Checking targets against current locks 2023-08-11T13:59:10.349 DEBUG:teuthology.nuke:shortname: smithi118 2023-08-11T13:59:10.349 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-11T13:59:10.371 DEBUG:teuthology.nuke:shortname: smithi183 2023-08-11T13:59:10.371 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-11T13:59:10.375 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi118.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365916', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 13:27:27.793635', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEBVjuVUDZ6IQwGnNoRxSjyFg+QP+93Ln+wR04DFtsjPEaS7/qbmS/B8yyVzFrUpHeNdC3gUF1BfKuMXgedbT2w='} 2023-08-11T13:59:10.401 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365916', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 13:27:27.795297', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCaEshNWMn+6Qm8csuH/Td6pNwne2cwfw0pk46xVWBvwKR/KIrQ3ch3p8aBpyRttc9wI5WF+S2tf+X8z4OJ5QYQVzEJCV9o+spSeJsW7yzQL93/pj+OwxF7DHSO4b1Lv/Qcy7cCWBfjD6DQXrNI5pTPh0wKblVguQxGHjwEaLKxs/XeXg1p715moHCpQio5SrT2CJv7oGnYLtfjn0A0uLqn+0V+hUC8boQSMQBq9OUH2pY3ldtDuNAIPEhnd9QT7oVIYBQhq9gQgHEK+UoHpQuorwII+3t8Bxu6JdKUajVjYYNLc0nM07m6PN2nswhYqCKvCSEQ8Wu9ERPQb03E3Dl/5IZywluRLtoQggz9TySb2mDyoUBQz7j3ixSh4CyYfFJUwRIdHPsOaRoUCfYfWKdgyEUOvt5AcfWFlBs4OZ6WN5IHX/sS4rjcLj02PIuF8wxDj11QVhoi+zGAasuw7Rz/HUHMT+hg6A02WDkCjsMqgnxqbXRS/COZpT7pa9HbyuE='} 2023-08-11T13:59:10.429 INFO:teuthology.orchestra.console.smithi118:Power off 2023-08-11T13:59:10.430 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-11T13:59:10.454 DEBUG:teuthology.orchestra.console.smithi118:power off output: Chassis Power Control: Down/Off 2023-08-11T13:59:10.456 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T13:59:10.475 INFO:teuthology.orchestra.console.smithi183:Power off 2023-08-11T13:59:10.476 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-11T13:59:10.489 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2023-08-11T13:59:10.500 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off 2023-08-11T13:59:10.501 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T13:59:10.521 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2023-08-11T13:59:10.590 INFO:teuthology.orchestra.console.smithi118:Power off completed 2023-08-11T13:59:10.623 INFO:teuthology.orchestra.console.smithi183:Power off completed 2023-08-11T13:59:10.749 INFO:teuthology.lock.ops:unlocked: smithi118.front.sepia.ceph.com 2023-08-11T13:59:10.765 INFO:teuthology.lock.ops:unlocked: smithi183.front.sepia.ceph.com