2023-09-13T04:46:55.231 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-13T04:46:55.233 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T04:46:55.396 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0575bf7d30>, signals=[15]) 2023-09-13T04:46:55.397 INFO:teuthology.lock.ops:Start node 'smithi130.front.sepia.ceph.com' reimaging 2023-09-13T04:46:55.397 INFO:teuthology.lock.ops:Updating [smithi130.front.sepia.ceph.com]: reset os type and version on server 2023-09-13T04:46:55.398 INFO:teuthology.lock.ops:Updating smithi130.front.sepia.ceph.com on lock server 2023-09-13T04:46:55.415 INFO:teuthology.lock.ops:Start node 'smithi164.front.sepia.ceph.com' reimaging 2023-09-13T04:46:55.416 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: reset os type and version on server 2023-09-13T04:46:55.416 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2023-09-13T04:46:55.417 DEBUG:teuthology.provision.fog.smithi130:FOG request: GET /host {"name": "smithi130"} 2023-09-13T04:46:55.440 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /host {"name": "smithi164"} 2023-09-13T04:46:55.488 DEBUG:teuthology.provision.fog.smithi130:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-09-13T04:46:55.507 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-09-13T04:46:55.529 DEBUG:teuthology.provision.fog.smithi130:Requesting image smithi_centos_8.stream (ID 3) 2023-09-13T04:46:55.529 DEBUG:teuthology.provision.fog.smithi130:FOG request: PUT /host/235 {"imageID": 3} 2023-09-13T04:46:55.548 DEBUG:teuthology.provision.fog.smithi164:Requesting image smithi_centos_8.stream (ID 3) 2023-09-13T04:46:55.549 DEBUG:teuthology.provision.fog.smithi164:FOG request: PUT /host/269 {"imageID": 3} 2023-09-13T04:46:55.568 INFO:teuthology.provision.fog.smithi130:Scheduling deploy of centos 8.stream 2023-09-13T04:46:55.569 DEBUG:teuthology.provision.fog.smithi130:FOG request: GET /task/active None 2023-09-13T04:46:55.594 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of centos 8.stream 2023-09-13T04:46:55.595 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-09-13T04:46:55.621 DEBUG:teuthology.provision.fog.smithi130:FOG request: GET /tasktype {"name": "deploy"} 2023-09-13T04:46:55.641 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /tasktype {"name": "deploy"} 2023-09-13T04:46:55.661 DEBUG:teuthology.provision.fog.smithi130:FOG request: POST /host/235/task {"taskTypeID": 1} 2023-09-13T04:46:55.684 DEBUG:teuthology.provision.fog.smithi164:FOG request: POST /host/269/task {"taskTypeID": 1} 2023-09-13T04:46:55.717 DEBUG:teuthology.provision.fog.smithi130:FOG request: GET /task/active None 2023-09-13T04:46:55.747 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-09-13T04:46:55.796 INFO:teuthology.orchestra.console.smithi130:Power off 2023-09-13T04:46:55.797 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-13T04:46:55.818 DEBUG:teuthology.orchestra.console.smithi130:power off output: Chassis Power Control: Down/Off 2023-09-13T04:46:55.821 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:46:55.836 INFO:teuthology.orchestra.console.smithi164:Power off 2023-09-13T04:46:55.837 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-13T04:46:55.844 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is on 2023-09-13T04:46:55.857 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off 2023-09-13T04:46:55.857 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:46:55.879 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2023-09-13T04:46:59.850 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:46:59.881 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:46:59.959 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is on 2023-09-13T04:46:59.989 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2023-09-13T04:47:03.962 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:47:03.990 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:47:04.071 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is off 2023-09-13T04:47:04.101 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2023-09-13T04:47:04.172 INFO:teuthology.orchestra.console.smithi130:Power off completed 2023-09-13T04:47:04.202 INFO:teuthology.orchestra.console.smithi164:Power off completed 2023-09-13T04:47:04.273 INFO:teuthology.orchestra.console.smithi130:Power on 2023-09-13T04:47:04.274 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-13T04:47:04.294 DEBUG:teuthology.orchestra.console.smithi130:power on output: Chassis Power Control: Up/On 2023-09-13T04:47:04.294 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:47:04.303 INFO:teuthology.orchestra.console.smithi164:Power on 2023-09-13T04:47:04.304 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-13T04:47:04.314 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is off 2023-09-13T04:47:04.323 DEBUG:teuthology.orchestra.console.smithi164:power on output: Chassis Power Control: Up/On 2023-09-13T04:47:04.323 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:47:04.342 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2023-09-13T04:47:08.318 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:47:08.343 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:47:08.427 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is off 2023-09-13T04:47:08.451 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2023-09-13T04:47:12.430 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:47:12.452 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T04:47:12.560 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2023-09-13T04:47:12.602 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is on 2023-09-13T04:47:12.661 INFO:teuthology.orchestra.console.smithi164:Power on completed 2023-09-13T04:47:12.703 INFO:teuthology.orchestra.console.smithi130:Power on completed 2023-09-13T04:47:12.763 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish 2023-09-13T04:47:12.763 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-09-13T04:47:12.804 INFO:teuthology.provision.fog.smithi130:Waiting for deploy to finish 2023-09-13T04:47:12.805 DEBUG:teuthology.provision.fog.smithi130:FOG request: GET /task/active None 2023-09-13T04:47:27.838 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-09-13T04:47:27.855 DEBUG:teuthology.provision.fog.smithi130:FOG request: GET /task/active None 2023-09-13T04:47:42.914 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-09-13T04:47:42.915 DEBUG:teuthology.provision.fog.smithi130:FOG request: GET /task/active None 2023-09-13T04:47:57.982 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-09-13T04:47:57.983 DEBUG:teuthology.provision.fog.smithi130:FOG request: GET /task/active None 2023-09-13T04:48:13.050 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-09-13T04:48:13.052 DEBUG:teuthology.provision.fog.smithi130:FOG request: GET /task/active None 2023-09-13T04:48:28.122 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-09-13T04:48:28.123 DEBUG:teuthology.provision.fog.smithi130:FOG request: GET /task/active None 2023-09-13T04:48:43.191 DEBUG:teuthology.provision.fog.smithi130:FOG request: GET /task/active None 2023-09-13T04:48:43.193 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-09-13T04:48:58.262 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-09-13T04:48:58.264 DEBUG:teuthology.provision.fog.smithi130:FOG request: GET /task/active None 2023-09-13T04:49:13.338 DEBUG:teuthology.provision.fog.smithi130:FOG request: GET /task/active None 2023-09-13T04:49:13.339 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-09-13T04:49:28.422 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-09-13T04:49:28.423 DEBUG:teuthology.provision.fog.smithi130:FOG request: GET /task/active None 2023-09-13T04:49:43.490 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-09-13T04:49:43.491 DEBUG:teuthology.provision.fog.smithi130:FOG request: GET /task/active None 2023-09-13T04:49:58.559 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None 2023-09-13T04:49:58.561 DEBUG:teuthology.provision.fog.smithi130:FOG request: GET /task/active None 2023-09-13T04:49:58.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:49:58.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:50:17.118 WARNING:teuthology.provision.fog.smithi130:[Errno None] Unable to connect to port 22 on 172.21.15.130 2023-09-13T04:50:17.119 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164 2023-09-13T04:50:23.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:50:23.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:50:26.202 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164 2023-09-13T04:50:26.203 WARNING:teuthology.provision.fog.smithi130:[Errno None] Unable to connect to port 22 on 172.21.15.130 2023-09-13T04:50:32.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:50:32.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:50:35.290 WARNING:teuthology.provision.fog.smithi130:[Errno None] Unable to connect to port 22 on 172.21.15.130 2023-09-13T04:50:35.291 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164 2023-09-13T04:50:41.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:50:41.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:50:44.346 WARNING:teuthology.provision.fog.smithi130:[Errno None] Unable to connect to port 22 on 172.21.15.130 2023-09-13T04:50:44.347 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164 2023-09-13T04:50:50.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:50:50.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:50:53.434 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164 2023-09-13T04:50:59.441 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:51:22.090 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-13T04:51:22.091 DEBUG:teuthology.orchestra.run.smithi130:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-13T04:51:47.432 INFO:teuthology.provision.fog.smithi130:Node is ready 2023-09-13T04:51:47.433 DEBUG:teuthology.orchestra.run.smithi130:> hostname 2023-09-13T04:51:47.497 INFO:teuthology.orchestra.run.smithi130.stdout:smithi130.front.sepia.ceph.com 2023-09-13T04:51:47.497 DEBUG:teuthology.orchestra.run.smithi130:> grep smithi130.front.sepia.ceph.com /etc/hosts 2023-09-13T04:51:47.558 INFO:teuthology.orchestra.run.smithi130.stdout:172.21.15.130 smithi130.front.sepia.ceph.com smithi130 2023-09-13T04:51:47.558 DEBUG:teuthology.orchestra.run.smithi130:> sudo hostname smithi130 2023-09-13T04:51:47.677 DEBUG:teuthology.orchestra.run.smithi130:> sudo sed -i -e 's/smithi130.front.sepia.ceph.com/smithi130/g' /etc/hosts 2023-09-13T04:51:47.707 DEBUG:teuthology.orchestra.run.smithi130:> sudo sed -i -e 's/172.21.15.130/172.21.15.130/g' /etc/hosts 2023-09-13T04:51:47.776 DEBUG:teuthology.orchestra.run.smithi130:> sudo sed -i -e 's/smithi130.front.sepia.ceph.com/smithi130/g' /etc/hostname 2023-09-13T04:51:47.844 DEBUG:teuthology.orchestra.run.smithi130:> sudo hostname smithi130 2023-09-13T04:51:47.911 DEBUG:teuthology.orchestra.run.smithi130:> cat /etc/os-release 2023-09-13T04:51:47.969 INFO:teuthology.orchestra.run.smithi130.stdout:NAME="CentOS Stream" 2023-09-13T04:51:47.970 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION="8" 2023-09-13T04:51:47.970 INFO:teuthology.orchestra.run.smithi130.stdout:ID="centos" 2023-09-13T04:51:47.970 INFO:teuthology.orchestra.run.smithi130.stdout:ID_LIKE="rhel fedora" 2023-09-13T04:51:47.970 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION_ID="8" 2023-09-13T04:51:47.971 INFO:teuthology.orchestra.run.smithi130.stdout:PLATFORM_ID="platform:el8" 2023-09-13T04:51:47.971 INFO:teuthology.orchestra.run.smithi130.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-13T04:51:47.971 INFO:teuthology.orchestra.run.smithi130.stdout:ANSI_COLOR="0;31" 2023-09-13T04:51:47.971 INFO:teuthology.orchestra.run.smithi130.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-13T04:51:47.972 INFO:teuthology.orchestra.run.smithi130.stdout:HOME_URL="https://centos.org/" 2023-09-13T04:51:47.972 INFO:teuthology.orchestra.run.smithi130.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T04:51:47.972 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T04:51:47.972 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-13T04:51:47.973 INFO:teuthology.provision.fog.smithi130:Deploy complete! 2023-09-13T04:51:47.974 DEBUG:teuthology.parallel:result is None 2023-09-13T04:51:59.454 WARNING:teuthology.provision.fog.smithi164:timed out 2023-09-13T04:52:05.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:52:05.668 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-13T04:52:05.668 DEBUG:teuthology.orchestra.run.smithi164:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-13T04:52:05.937 INFO:teuthology.provision.fog.smithi164:Node is ready 2023-09-13T04:52:05.938 DEBUG:teuthology.orchestra.run.smithi164:> hostname 2023-09-13T04:52:05.957 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164.front.sepia.ceph.com 2023-09-13T04:52:05.957 DEBUG:teuthology.orchestra.run.smithi164:> grep smithi164.front.sepia.ceph.com /etc/hosts 2023-09-13T04:52:06.016 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164 smithi164.front.sepia.ceph.com smithi164 2023-09-13T04:52:06.017 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164 2023-09-13T04:52:06.110 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164.front.sepia.ceph.com/smithi164/g' /etc/hosts 2023-09-13T04:52:06.140 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/172.21.15.164/172.21.15.164/g' /etc/hosts 2023-09-13T04:52:06.210 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164.front.sepia.ceph.com/smithi164/g' /etc/hostname 2023-09-13T04:52:06.279 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164 2023-09-13T04:52:06.347 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2023-09-13T04:52:06.406 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2023-09-13T04:52:06.406 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="8" 2023-09-13T04:52:06.407 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2023-09-13T04:52:06.407 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2023-09-13T04:52:06.407 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="8" 2023-09-13T04:52:06.408 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el8" 2023-09-13T04:52:06.408 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-13T04:52:06.408 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2023-09-13T04:52:06.408 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-13T04:52:06.408 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2023-09-13T04:52:06.408 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T04:52:06.409 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T04:52:06.409 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-13T04:52:06.410 INFO:teuthology.provision.fog.smithi164:Deploy complete! 2023-09-13T04:52:06.410 DEBUG:teuthology.parallel:result is None 2023-09-13T04:52:06.597 INFO:teuthology.lock.ops:Checking smithi130.front.sepia.ceph.com 2023-09-13T04:52:06.597 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-13T04:52:06.616 INFO:teuthology.lock.ops:Checking smithi164.front.sepia.ceph.com 2023-09-13T04:52:06.616 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-13T04:52:06.634 INFO:teuthology.lock.ops:Updating [smithi130.front.sepia.ceph.com]: set os type and version on server 2023-09-13T04:52:06.635 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2023-09-13T04:52:06.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:52:06.702 DEBUG:teuthology.orchestra.run.smithi130:> true 2023-09-13T04:52:06.837 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi130.front.sepia.ceph.com' 2023-09-13T04:52:06.837 DEBUG:teuthology.orchestra.run.smithi130:> uname -m 2023-09-13T04:52:06.895 INFO:teuthology.orchestra.run.smithi130.stdout:x86_64 2023-09-13T04:52:06.896 DEBUG:teuthology.orchestra.run.smithi130:> cat /etc/os-release 2023-09-13T04:52:06.953 INFO:teuthology.orchestra.run.smithi130.stdout:NAME="CentOS Stream" 2023-09-13T04:52:06.954 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION="8" 2023-09-13T04:52:06.954 INFO:teuthology.orchestra.run.smithi130.stdout:ID="centos" 2023-09-13T04:52:06.954 INFO:teuthology.orchestra.run.smithi130.stdout:ID_LIKE="rhel fedora" 2023-09-13T04:52:06.954 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION_ID="8" 2023-09-13T04:52:06.954 INFO:teuthology.orchestra.run.smithi130.stdout:PLATFORM_ID="platform:el8" 2023-09-13T04:52:06.955 INFO:teuthology.orchestra.run.smithi130.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-13T04:52:06.955 INFO:teuthology.orchestra.run.smithi130.stdout:ANSI_COLOR="0;31" 2023-09-13T04:52:06.955 INFO:teuthology.orchestra.run.smithi130.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-13T04:52:06.955 INFO:teuthology.orchestra.run.smithi130.stdout:HOME_URL="https://centos.org/" 2023-09-13T04:52:06.955 INFO:teuthology.orchestra.run.smithi130.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T04:52:06.956 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T04:52:06.956 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-13T04:52:06.956 INFO:teuthology.lock.ops:Updating smithi130.front.sepia.ceph.com on lock server 2023-09-13T04:52:06.975 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: set os type and version on server 2023-09-13T04:52:06.975 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2023-09-13T04:52:06.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T04:52:07.040 DEBUG:teuthology.orchestra.run.smithi164:> true 2023-09-13T04:52:07.112 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2023-09-13T04:52:07.112 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2023-09-13T04:52:07.170 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2023-09-13T04:52:07.170 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2023-09-13T04:52:07.228 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2023-09-13T04:52:07.228 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="8" 2023-09-13T04:52:07.228 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2023-09-13T04:52:07.229 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2023-09-13T04:52:07.229 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="8" 2023-09-13T04:52:07.229 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el8" 2023-09-13T04:52:07.229 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-13T04:52:07.229 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2023-09-13T04:52:07.229 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-13T04:52:07.230 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2023-09-13T04:52:07.230 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T04:52:07.230 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T04:52:07.230 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-13T04:52:07.231 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2023-09-13T04:52:07.258 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T04:52:07.361 INFO:teuthology.dispatcher.supervisor:Running job 7394748 2023-09-13T04:52:07.361 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394748 --name rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394748/orig.config.yaml 2023-09-13T04:52:07.365 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394748 2023-09-13T04:52:07.366 INFO:teuthology.dispatcher.supervisor:Job PID: 3796382 2023-09-13T04:52:07.366 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-09-13T04:54:07.467 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T04:56:07.643 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T04:58:08.127 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T05:00:08.291 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T05:02:08.459 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T05:04:08.623 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T05:06:08.815 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T05:08:08.979 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T05:10:09.143 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T05:12:09.311 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T05:14:09.471 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T05:16:09.639 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T05:16:09.722 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-09-13T05:16:09.902 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-09-13T05:16:09.902 INFO:teuthology.nuke:Checking targets against current locks 2023-09-13T05:16:09.949 DEBUG:teuthology.nuke:shortname: smithi130 2023-09-13T05:16:09.950 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-13T05:16:09.968 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi130.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394748', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 04:46:54.423843', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE8yPUX+wGxvA1iZSEI1+xz/v4g/2Fzw6eytKKgglQrzVCXZkvgC0DKLveC6GBPojfYxef/1J6ScQgjMKK5Nfpk='} 2023-09-13T05:16:09.971 DEBUG:teuthology.nuke:shortname: smithi164 2023-09-13T05:16:09.972 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-13T05:16:09.995 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-09-12_12:12:15-fs-wip-rishabh-2023sep12-b2-testing-default-smithi/7394748', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 04:46:54.422299', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGhsekz81UsZfNSBG4wq9gVX+mkU0WqpRVOncXtCS5eMsahT84T7vgn7RCa1yMUkGnz7/kEhWkekMlx6PSh1R4U='} 2023-09-13T05:16:10.022 INFO:teuthology.orchestra.console.smithi130:Power off 2023-09-13T05:16:10.022 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-13T05:16:10.039 INFO:teuthology.orchestra.console.smithi164:Power off 2023-09-13T05:16:10.040 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-13T05:16:10.047 DEBUG:teuthology.orchestra.console.smithi130:power off output: Chassis Power Control: Down/Off 2023-09-13T05:16:10.048 DEBUG:teuthology.orchestra.console.smithi130:pexpect command: ipmitool -H smithi130.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T05:16:10.059 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off 2023-09-13T05:16:10.060 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T05:16:10.067 DEBUG:teuthology.orchestra.console.smithi130:check power output: Chassis Power is off 2023-09-13T05:16:10.080 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2023-09-13T05:16:10.168 INFO:teuthology.orchestra.console.smithi130:Power off completed 2023-09-13T05:16:10.181 INFO:teuthology.orchestra.console.smithi164:Power off completed 2023-09-13T05:16:10.308 INFO:teuthology.lock.ops:unlocked: smithi130.front.sepia.ceph.com 2023-09-13T05:16:10.325 INFO:teuthology.lock.ops:unlocked: smithi164.front.sepia.ceph.com