2023-08-31T05:11:24.134 INFO:root:teuthology version: 0.0.1.dev192+gf9a8556 2023-08-31T05:11:24.135 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:11:24.288 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f483c3b1670>, signals=[15]) 2023-08-31T05:11:24.289 INFO:teuthology.lock.ops:Start node 'smithi137.front.sepia.ceph.com' reimaging 2023-08-31T05:11:24.290 INFO:teuthology.lock.ops:Updating [smithi137.front.sepia.ceph.com]: reset os type and version on server 2023-08-31T05:11:24.290 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2023-08-31T05:11:24.315 INFO:teuthology.lock.ops:Start node 'smithi188.front.sepia.ceph.com' reimaging 2023-08-31T05:11:24.316 INFO:teuthology.lock.ops:Updating [smithi188.front.sepia.ceph.com]: reset os type and version on server 2023-08-31T05:11:24.316 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2023-08-31T05:11:24.318 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /host {"name": "smithi137"} 2023-08-31T05:11:24.341 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /host {"name": "smithi188"} 2023-08-31T05:11:24.406 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-08-31T05:11:24.413 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-08-31T05:11:24.450 DEBUG:teuthology.provision.fog.smithi137:Requesting image smithi_centos_8.stream (ID 3) 2023-08-31T05:11:24.450 DEBUG:teuthology.provision.fog.smithi137:FOG request: PUT /host/242 {"imageID": 3} 2023-08-31T05:11:24.455 DEBUG:teuthology.provision.fog.smithi188:Requesting image smithi_centos_8.stream (ID 3) 2023-08-31T05:11:24.455 DEBUG:teuthology.provision.fog.smithi188:FOG request: PUT /host/293 {"imageID": 3} 2023-08-31T05:11:24.492 INFO:teuthology.provision.fog.smithi137:Scheduling deploy of centos 8.stream 2023-08-31T05:11:24.492 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-08-31T05:11:24.501 INFO:teuthology.provision.fog.smithi188:Scheduling deploy of centos 8.stream 2023-08-31T05:11:24.502 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-08-31T05:11:24.540 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /tasktype {"name": "deploy"} 2023-08-31T05:11:24.545 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /tasktype {"name": "deploy"} 2023-08-31T05:11:24.576 DEBUG:teuthology.provision.fog.smithi188:FOG request: POST /host/293/task {"taskTypeID": 1} 2023-08-31T05:11:24.580 DEBUG:teuthology.provision.fog.smithi137:FOG request: POST /host/242/task {"taskTypeID": 1} 2023-08-31T05:11:24.629 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-08-31T05:11:24.632 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-08-31T05:11:24.718 INFO:teuthology.orchestra.console.smithi137:Power off 2023-08-31T05:11:24.719 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-31T05:11:24.741 INFO:teuthology.orchestra.console.smithi188:Power off 2023-08-31T05:11:24.742 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-31T05:11:24.749 DEBUG:teuthology.orchestra.console.smithi137:power off output: Chassis Power Control: Down/Off 2023-08-31T05:11:24.750 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-31T05:11:24.762 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off 2023-08-31T05:11:24.763 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-31T05:11:24.775 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2023-08-31T05:11:24.785 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2023-08-31T05:11:28.778 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-31T05:11:28.786 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-31T05:11:28.887 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2023-08-31T05:11:28.896 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2023-08-31T05:11:32.890 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-31T05:11:32.897 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-31T05:11:32.999 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2023-08-31T05:11:33.005 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2023-08-31T05:11:37.002 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-31T05:11:37.010 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-31T05:11:37.112 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off 2023-08-31T05:11:37.118 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2023-08-31T05:11:37.214 INFO:teuthology.orchestra.console.smithi137:Power off completed 2023-08-31T05:11:37.220 INFO:teuthology.orchestra.console.smithi188:Power off completed 2023-08-31T05:11:37.315 INFO:teuthology.orchestra.console.smithi137:Power on 2023-08-31T05:11:37.316 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-31T05:11:37.324 INFO:teuthology.orchestra.console.smithi188:Power on 2023-08-31T05:11:37.324 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-31T05:11:37.337 DEBUG:teuthology.orchestra.console.smithi137:power on output: Chassis Power Control: Up/On 2023-08-31T05:11:37.338 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-31T05:11:37.348 DEBUG:teuthology.orchestra.console.smithi188:power on output: Chassis Power Control: Up/On 2023-08-31T05:11:37.349 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-31T05:11:37.360 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off 2023-08-31T05:11:37.369 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2023-08-31T05:11:41.362 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-31T05:11:41.370 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-31T05:11:41.471 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2023-08-31T05:11:41.479 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2023-08-31T05:11:41.573 INFO:teuthology.orchestra.console.smithi137:Power on completed 2023-08-31T05:11:41.580 INFO:teuthology.orchestra.console.smithi188:Power on completed 2023-08-31T05:11:41.674 INFO:teuthology.provision.fog.smithi137:Waiting for deploy to finish 2023-08-31T05:11:41.675 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-08-31T05:11:41.681 INFO:teuthology.provision.fog.smithi188:Waiting for deploy to finish 2023-08-31T05:11:41.682 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-08-31T05:11:56.750 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-08-31T05:11:56.752 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-08-31T05:12:11.822 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-08-31T05:12:11.824 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-08-31T05:12:26.894 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-08-31T05:12:26.895 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-08-31T05:12:41.970 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-08-31T05:12:41.972 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-08-31T05:12:57.048 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-08-31T05:12:57.050 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-08-31T05:13:12.120 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-08-31T05:13:12.122 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-08-31T05:13:27.196 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-08-31T05:13:27.197 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-08-31T05:13:42.266 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-08-31T05:13:42.267 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-08-31T05:13:57.336 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-08-31T05:13:57.338 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-08-31T05:14:12.410 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-08-31T05:14:12.411 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-08-31T05:14:27.482 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-08-31T05:14:27.484 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None 2023-08-31T05:14:27.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-31T05:14:27.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-31T05:14:30.618 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137 2023-08-31T05:14:30.619 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-08-31T05:14:36.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-31T05:14:36.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-31T05:14:39.674 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137 2023-08-31T05:14:39.675 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-08-31T05:14:45.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-31T05:14:45.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-31T05:14:48.730 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137 2023-08-31T05:14:48.762 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-08-31T05:14:54.734 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-31T05:14:54.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-31T05:14:57.786 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137 2023-08-31T05:14:57.818 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-08-31T05:15:03.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-31T05:15:03.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-31T05:15:06.842 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137 2023-08-31T05:15:06.878 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-08-31T05:15:12.849 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-31T05:15:12.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-31T05:15:15.931 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-08-31T05:15:21.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-31T05:16:12.902 WARNING:teuthology.provision.fog.smithi137:timed out 2023-08-31T05:16:18.907 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-31T05:16:19.273 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-31T05:16:19.274 DEBUG:teuthology.orchestra.run.smithi137:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-31T05:16:19.500 INFO:teuthology.provision.fog.smithi137:Node is ready 2023-08-31T05:16:19.501 DEBUG:teuthology.orchestra.run.smithi137:> hostname 2023-08-31T05:16:19.560 INFO:teuthology.orchestra.run.smithi137.stdout:smithi137.front.sepia.ceph.com 2023-08-31T05:16:19.561 DEBUG:teuthology.orchestra.run.smithi137:> grep smithi137.front.sepia.ceph.com /etc/hosts 2023-08-31T05:16:19.619 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137 smithi137.front.sepia.ceph.com smithi137 2023-08-31T05:16:19.620 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname smithi137 2023-08-31T05:16:19.711 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i -e 's/smithi137.front.sepia.ceph.com/smithi137/g' /etc/hosts 2023-08-31T05:16:19.742 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i -e 's/172.21.15.137/172.21.15.137/g' /etc/hosts 2023-08-31T05:16:19.810 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i -e 's/smithi137.front.sepia.ceph.com/smithi137/g' /etc/hostname 2023-08-31T05:16:19.879 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname smithi137 2023-08-31T05:16:19.947 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2023-08-31T05:16:20.006 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2023-08-31T05:16:20.006 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="8" 2023-08-31T05:16:20.007 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2023-08-31T05:16:20.007 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2023-08-31T05:16:20.008 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="8" 2023-08-31T05:16:20.008 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el8" 2023-08-31T05:16:20.008 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-31T05:16:20.009 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2023-08-31T05:16:20.009 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-31T05:16:20.009 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2023-08-31T05:16:20.009 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-31T05:16:20.010 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-31T05:16:20.010 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-31T05:16:20.011 INFO:teuthology.provision.fog.smithi137:Deploy complete! 2023-08-31T05:16:20.012 DEBUG:teuthology.parallel:result is None 2023-08-31T05:16:21.942 WARNING:teuthology.provision.fog.smithi188:timed out 2023-08-31T05:16:27.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-31T05:16:28.158 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-31T05:16:28.159 DEBUG:teuthology.orchestra.run.smithi188:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-31T05:16:28.410 INFO:teuthology.provision.fog.smithi188:Node is ready 2023-08-31T05:16:28.411 DEBUG:teuthology.orchestra.run.smithi188:> hostname 2023-08-31T05:16:28.430 INFO:teuthology.orchestra.run.smithi188.stdout:smithi188.front.sepia.ceph.com 2023-08-31T05:16:28.430 DEBUG:teuthology.orchestra.run.smithi188:> grep smithi188.front.sepia.ceph.com /etc/hosts 2023-08-31T05:16:28.488 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.15.188 smithi188.front.sepia.ceph.com smithi188 2023-08-31T05:16:28.489 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname smithi188 2023-08-31T05:16:28.597 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i -e 's/smithi188.front.sepia.ceph.com/smithi188/g' /etc/hosts 2023-08-31T05:16:28.625 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i -e 's/172.21.15.188/172.21.15.188/g' /etc/hosts 2023-08-31T05:16:28.692 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i -e 's/smithi188.front.sepia.ceph.com/smithi188/g' /etc/hostname 2023-08-31T05:16:28.760 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname smithi188 2023-08-31T05:16:28.828 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2023-08-31T05:16:28.886 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2023-08-31T05:16:28.887 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="8" 2023-08-31T05:16:28.887 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2023-08-31T05:16:28.888 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2023-08-31T05:16:28.888 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="8" 2023-08-31T05:16:28.888 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el8" 2023-08-31T05:16:28.888 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-31T05:16:28.888 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2023-08-31T05:16:28.889 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-31T05:16:28.889 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2023-08-31T05:16:28.889 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-31T05:16:28.889 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-31T05:16:28.889 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-31T05:16:28.890 INFO:teuthology.provision.fog.smithi188:Deploy complete! 2023-08-31T05:16:28.891 DEBUG:teuthology.parallel:result is None 2023-08-31T05:16:29.051 INFO:teuthology.lock.ops:Checking smithi137.front.sepia.ceph.com 2023-08-31T05:16:29.052 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-31T05:16:29.077 INFO:teuthology.lock.ops:Checking smithi188.front.sepia.ceph.com 2023-08-31T05:16:29.077 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-31T05:16:29.102 INFO:teuthology.lock.ops:Updating [smithi137.front.sepia.ceph.com]: set os type and version on server 2023-08-31T05:16:29.103 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2023-08-31T05:16:29.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-31T05:16:29.172 DEBUG:teuthology.orchestra.run.smithi137:> true 2023-08-31T05:16:29.244 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2023-08-31T05:16:29.244 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2023-08-31T05:16:29.306 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2023-08-31T05:16:29.307 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2023-08-31T05:16:29.366 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2023-08-31T05:16:29.367 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="8" 2023-08-31T05:16:29.367 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2023-08-31T05:16:29.367 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2023-08-31T05:16:29.367 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="8" 2023-08-31T05:16:29.367 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el8" 2023-08-31T05:16:29.368 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-31T05:16:29.368 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2023-08-31T05:16:29.368 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-31T05:16:29.368 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2023-08-31T05:16:29.369 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-31T05:16:29.369 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-31T05:16:29.369 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-31T05:16:29.370 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2023-08-31T05:16:29.418 INFO:teuthology.lock.ops:Updating [smithi188.front.sepia.ceph.com]: set os type and version on server 2023-08-31T05:16:29.419 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-08-31T05:16:29.421 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-31T05:16:29.487 DEBUG:teuthology.orchestra.run.smithi188:> true 2023-08-31T05:16:29.553 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-08-31T05:16:29.553 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2023-08-31T05:16:29.611 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2023-08-31T05:16:29.612 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2023-08-31T05:16:29.670 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2023-08-31T05:16:29.670 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="8" 2023-08-31T05:16:29.671 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2023-08-31T05:16:29.671 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2023-08-31T05:16:29.671 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="8" 2023-08-31T05:16:29.671 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el8" 2023-08-31T05:16:29.671 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-31T05:16:29.672 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2023-08-31T05:16:29.672 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-31T05:16:29.672 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2023-08-31T05:16:29.672 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-31T05:16:29.672 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-31T05:16:29.673 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-31T05:16:29.673 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2023-08-31T05:16:29.702 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:16:29.805 INFO:teuthology.dispatcher.supervisor:Running job 7385275 2023-08-31T05:16:29.805 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-08-31_04:13:51-fs-wip-vshankar-testing-20230830.153114-testing-default-smithi/7385275 --name vshankar-2023-08-31_04:13:51-fs-wip-vshankar-testing-20230830.153114-testing-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/vshankar-2023-08-31_04:13:51-fs-wip-vshankar-testing-20230830.153114-testing-default-smithi/7385275/orig.config.yaml 2023-08-31T05:16:29.808 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2023-08-31_04:13:51-fs-wip-vshankar-testing-20230830.153114-testing-default-smithi/7385275 2023-08-31T05:16:29.809 INFO:teuthology.dispatcher.supervisor:Job PID: 2900444 2023-08-31T05:16:29.809 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-08-31T05:18:29.889 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:20:30.063 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:22:30.239 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:24:30.411 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:26:30.579 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:28:30.755 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:30:30.943 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:32:31.111 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:34:31.279 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:36:31.455 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:38:31.630 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:40:31.795 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:42:32.083 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:44:32.255 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:46:32.423 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:48:32.575 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:50:32.743 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:52:32.923 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:54:33.099 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:56:33.300 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:58:33.455 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-31T05:58:33.524 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-08-31T05:58:33.738 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-08-31T05:58:33.739 INFO:teuthology.nuke:Checking targets against current locks 2023-08-31T05:58:33.788 DEBUG:teuthology.nuke:shortname: smithi137 2023-08-31T05:58:33.788 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-31T05:58:33.811 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-08-31_04:13:51-fs-wip-vshankar-testing-20230830.153114-testing-default-smithi/7385275', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-31 05:11:23.297559', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDagCuBTW2lUFEQG9mp6M0Zyado9f8vVdzu4cd/W2XpZH1vg6BHkTNKSlTDMXr9UloT4Xm6JlQ5q/Mm/bj9NKl6wmnQ36Ic0xYPCF0Xr1uesieyPUsL77y0ijeJqsrW4FLSwJzRNhH+5DLzDeX7Cg+24MbdQCwzB0MKHVvXyFt6G9pgugJj4lTz5CnRDu/bCPx83lSvmxi3qzNyU8tKXCuRx9ML0IdToDhtm6IiYTQhTgItaEPHJ6e4f4J2iUuB4ubQN9pl95ozHGq3tbZ1x+jX6ODzRfgP9T/joo3O8TNKCYbynh10680/hNMn9q8voxe7QBzMIrRqrrhrl/rP4tMF+CQA28LfA8m91fQngu2dqzc0N1G9uVPSu5yE6JNqnqQvjcTY8La8BjFgzZT7kwwXy+6oYOyQuGtRxF5LSADfSJOKv93Bb2nWFaBX6gvtgZKR9QVr8OhEhWs5J5wUkXj6IshDV4qeiWw/Ef8YmqQ26wpOkW3+5plFBcg5w0ZWUN0='} 2023-08-31T05:58:33.814 DEBUG:teuthology.nuke:shortname: smithi188 2023-08-31T05:58:33.814 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-31T05:58:33.837 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-08-31_04:13:51-fs-wip-vshankar-testing-20230830.153114-testing-default-smithi/7385275', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-31 05:11:23.299526', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDc/D9bJEeCDVYlLlbweY9CfhIalEvGJEx1LuMQNgBQsTHICE+FcXyeajMe7BUvhoHs+FMxoGKXABqNbn79XFgYLjMxVbZSYEQAnJ1a9ud3me5MuIysicGYbn/YouCTjB43rmMdOTrNj0BphOq2qizuqNfOelGG6OHy4NjyKihpi3kgXA0nZyd2fJZb7ASaAFXa7GvSnrkgy6IJ53NzbgLYniX3afgGKDh7JrWWtdFoZPYGqJntOQ6kJrtmoipz8VV+TMJ5gCSNUy26viQFgBUiCeFnm7BsQgYvg/44iQJPBqVHoM45Dhm+8hEhvwMNsyyIVZta339L7uXUdMunZkKk65KhzS6Oyygq9bZYujNbtgWaEp1oCA0p1UHpl08vLIdOV8tmRgu1qW8CE+YaUnZU4sZWxrmaKA/NewlKL3b9sdY1Lhu/IUHGL49+k990+pv8DXL6nMr2OfEEpu9/CeeiYtjWsHJJWuqsJAqza2msOBHMO6z9d2QNWgMxINowTkM='} 2023-08-31T05:58:33.866 INFO:teuthology.orchestra.console.smithi137:Power off 2023-08-31T05:58:33.866 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-31T05:58:33.886 DEBUG:teuthology.orchestra.console.smithi137:power off output: Chassis Power Control: Down/Off 2023-08-31T05:58:33.886 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-31T05:58:33.901 INFO:teuthology.orchestra.console.smithi188:Power off 2023-08-31T05:58:33.902 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-31T05:58:33.911 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off 2023-08-31T05:58:33.924 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off 2023-08-31T05:58:33.924 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-31T05:58:33.945 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2023-08-31T05:58:34.012 INFO:teuthology.orchestra.console.smithi137:Power off completed 2023-08-31T05:58:34.046 INFO:teuthology.orchestra.console.smithi188:Power off completed 2023-08-31T05:58:34.164 INFO:teuthology.lock.ops:unlocked: smithi137.front.sepia.ceph.com 2023-08-31T05:58:34.198 INFO:teuthology.lock.ops:unlocked: smithi188.front.sepia.ceph.com