2023-10-19T13:53:56.745 INFO:root:teuthology version: 0.0.1.dev201+g202b180 2023-10-19T13:53:56.747 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T13:53:56.876 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7feef5b9b670>, signals=[15]) 2023-10-19T13:53:56.877 INFO:teuthology.lock.ops:Start node 'smithi138.front.sepia.ceph.com' reimaging 2023-10-19T13:53:56.877 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: reset os type and version on server 2023-10-19T13:53:56.877 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2023-10-19T13:53:56.896 INFO:teuthology.lock.ops:Start node 'smithi190.front.sepia.ceph.com' reimaging 2023-10-19T13:53:56.896 INFO:teuthology.lock.ops:Updating [smithi190.front.sepia.ceph.com]: reset os type and version on server 2023-10-19T13:53:56.896 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2023-10-19T13:53:56.898 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /host {"name": "smithi138"} 2023-10-19T13:53:56.920 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /host {"name": "smithi190"} 2023-10-19T13:53:56.978 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-10-19T13:53:56.981 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-10-19T13:53:57.029 DEBUG:teuthology.provision.fog.smithi138:Requesting image smithi_centos_8.stream (ID 3) 2023-10-19T13:53:57.029 DEBUG:teuthology.provision.fog.smithi138:FOG request: PUT /host/243 {"imageID": 3} 2023-10-19T13:53:57.032 DEBUG:teuthology.provision.fog.smithi190:Requesting image smithi_centos_8.stream (ID 3) 2023-10-19T13:53:57.033 DEBUG:teuthology.provision.fog.smithi190:FOG request: PUT /host/295 {"imageID": 3} 2023-10-19T13:53:57.072 INFO:teuthology.provision.fog.smithi138:Scheduling deploy of centos 8.stream 2023-10-19T13:53:57.073 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-10-19T13:53:57.081 INFO:teuthology.provision.fog.smithi190:Scheduling deploy of centos 8.stream 2023-10-19T13:53:57.082 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None 2023-10-19T13:53:57.128 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /tasktype {"name": "deploy"} 2023-10-19T13:53:57.134 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /tasktype {"name": "deploy"} 2023-10-19T13:53:57.166 DEBUG:teuthology.provision.fog.smithi138:FOG request: POST /host/243/task {"taskTypeID": 1} 2023-10-19T13:53:57.172 DEBUG:teuthology.provision.fog.smithi190:FOG request: POST /host/295/task {"taskTypeID": 1} 2023-10-19T13:53:57.226 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None 2023-10-19T13:53:57.229 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-10-19T13:53:57.318 INFO:teuthology.orchestra.console.smithi138:Power off 2023-10-19T13:53:57.319 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-19T13:53:57.331 INFO:teuthology.orchestra.console.smithi190:Power off 2023-10-19T13:53:57.331 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-19T13:53:57.338 DEBUG:teuthology.orchestra.console.smithi138:power off output: Chassis Power Control: Down/Off 2023-10-19T13:53:57.339 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-19T13:53:57.351 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off 2023-10-19T13:53:57.351 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-19T13:53:57.358 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2023-10-19T13:53:57.371 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2023-10-19T13:53:57.459 INFO:teuthology.orchestra.console.smithi138:Power off completed 2023-10-19T13:53:57.472 INFO:teuthology.orchestra.console.smithi190:Power off completed 2023-10-19T13:53:57.560 INFO:teuthology.orchestra.console.smithi138:Power on 2023-10-19T13:53:57.561 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-19T13:53:57.572 INFO:teuthology.orchestra.console.smithi190:Power on 2023-10-19T13:53:57.573 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-19T13:53:57.584 DEBUG:teuthology.orchestra.console.smithi138:power on output: Chassis Power Control: Up/On 2023-10-19T13:53:57.586 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-19T13:53:57.596 DEBUG:teuthology.orchestra.console.smithi190:power on output: Chassis Power Control: Up/On 2023-10-19T13:53:57.596 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-19T13:53:57.606 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2023-10-19T13:53:57.616 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2023-10-19T13:54:01.610 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-19T13:54:01.618 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-19T13:54:01.719 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2023-10-19T13:54:01.726 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2023-10-19T13:54:05.722 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-19T13:54:05.729 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-19T13:54:05.830 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2023-10-19T13:54:05.838 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2023-10-19T13:54:05.932 INFO:teuthology.orchestra.console.smithi138:Power on completed 2023-10-19T13:54:05.939 INFO:teuthology.orchestra.console.smithi190:Power on completed 2023-10-19T13:54:06.033 INFO:teuthology.provision.fog.smithi138:Waiting for deploy to finish 2023-10-19T13:54:06.034 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-10-19T13:54:06.040 INFO:teuthology.provision.fog.smithi190:Waiting for deploy to finish 2023-10-19T13:54:06.041 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None 2023-10-19T13:54:21.113 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-10-19T13:54:21.115 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None 2023-10-19T13:54:36.190 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-10-19T13:54:36.192 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None 2023-10-19T13:54:51.271 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None 2023-10-19T13:54:51.273 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-10-19T13:55:06.348 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None 2023-10-19T13:55:06.349 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-10-19T13:55:21.423 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-10-19T13:55:21.424 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None 2023-10-19T13:55:36.503 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-10-19T13:55:36.504 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None 2023-10-19T13:55:51.586 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-10-19T13:55:51.588 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None 2023-10-19T13:56:06.670 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None 2023-10-19T13:56:06.672 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-10-19T13:56:21.751 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None 2023-10-19T13:56:21.752 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-10-19T13:56:36.826 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-10-19T13:56:36.828 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None 2023-10-19T13:56:51.894 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None 2023-10-19T13:56:51.895 DEBUG:teuthology.provision.fog.smithi138:FOG request: GET /task/active None 2023-10-19T13:56:51.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:56:51.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:57:10.394 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-10-19T13:57:10.395 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2023-10-19T13:57:16.402 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:57:16.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:57:19.482 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-10-19T13:57:19.483 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2023-10-19T13:57:25.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:57:25.492 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:57:28.570 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-10-19T13:57:28.571 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2023-10-19T13:57:34.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:57:34.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:57:37.626 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-10-19T13:57:37.658 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2023-10-19T13:57:43.630 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:57:43.660 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:57:46.718 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2023-10-19T13:57:49.722 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-10-19T13:57:52.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:57:55.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:57:55.802 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2023-10-19T13:57:58.782 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-10-19T13:58:01.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:58:04.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:58:07.866 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-10-19T13:58:13.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:58:15.474 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-19T13:58:15.475 DEBUG:teuthology.orchestra.run.smithi190:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-19T13:58:40.483 INFO:teuthology.provision.fog.smithi190:Node is ready 2023-10-19T13:58:40.483 DEBUG:teuthology.orchestra.run.smithi190:> hostname 2023-10-19T13:58:40.508 INFO:teuthology.orchestra.run.smithi190.stdout:smithi190.front.sepia.ceph.com 2023-10-19T13:58:40.508 DEBUG:teuthology.orchestra.run.smithi190:> grep smithi190.front.sepia.ceph.com /etc/hosts 2023-10-19T13:58:40.569 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.15.190 smithi190.front.sepia.ceph.com smithi190 2023-10-19T13:58:40.569 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname smithi190 2023-10-19T13:58:40.703 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i -e 's/smithi190.front.sepia.ceph.com/smithi190/g' /etc/hosts 2023-10-19T13:58:40.735 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i -e 's/172.21.15.190/172.21.15.190/g' /etc/hosts 2023-10-19T13:58:40.805 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i -e 's/smithi190.front.sepia.ceph.com/smithi190/g' /etc/hostname 2023-10-19T13:58:40.874 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname smithi190 2023-10-19T13:58:40.942 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2023-10-19T13:58:41.000 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream" 2023-10-19T13:58:41.001 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="8" 2023-10-19T13:58:41.001 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2023-10-19T13:58:41.001 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2023-10-19T13:58:41.001 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="8" 2023-10-19T13:58:41.002 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el8" 2023-10-19T13:58:41.002 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-19T13:58:41.002 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2023-10-19T13:58:41.002 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-19T13:58:41.003 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2023-10-19T13:58:41.003 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-19T13:58:41.003 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-19T13:58:41.003 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-19T13:58:41.004 INFO:teuthology.provision.fog.smithi190:Deploy complete! 2023-10-19T13:58:41.005 DEBUG:teuthology.parallel:result is None 2023-10-19T13:59:01.830 WARNING:teuthology.provision.fog.smithi138:timed out 2023-10-19T13:59:07.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:59:08.077 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-19T13:59:08.078 DEBUG:teuthology.orchestra.run.smithi138:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-19T13:59:08.308 INFO:teuthology.provision.fog.smithi138:Node is ready 2023-10-19T13:59:08.308 DEBUG:teuthology.orchestra.run.smithi138:> hostname 2023-10-19T13:59:08.326 INFO:teuthology.orchestra.run.smithi138.stdout:smithi138.front.sepia.ceph.com 2023-10-19T13:59:08.327 DEBUG:teuthology.orchestra.run.smithi138:> grep smithi138.front.sepia.ceph.com /etc/hosts 2023-10-19T13:59:08.385 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138 smithi138.front.sepia.ceph.com smithi138 2023-10-19T13:59:08.386 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname smithi138 2023-10-19T13:59:08.478 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i -e 's/smithi138.front.sepia.ceph.com/smithi138/g' /etc/hosts 2023-10-19T13:59:08.507 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i -e 's/172.21.15.138/172.21.15.138/g' /etc/hosts 2023-10-19T13:59:08.575 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i -e 's/smithi138.front.sepia.ceph.com/smithi138/g' /etc/hostname 2023-10-19T13:59:08.642 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname smithi138 2023-10-19T13:59:08.708 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2023-10-19T13:59:08.765 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2023-10-19T13:59:08.766 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="8" 2023-10-19T13:59:08.766 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2023-10-19T13:59:08.766 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2023-10-19T13:59:08.767 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="8" 2023-10-19T13:59:08.767 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el8" 2023-10-19T13:59:08.767 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-19T13:59:08.767 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2023-10-19T13:59:08.767 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-19T13:59:08.768 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2023-10-19T13:59:08.768 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-19T13:59:08.768 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-19T13:59:08.768 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-19T13:59:08.768 INFO:teuthology.provision.fog.smithi138:Deploy complete! 2023-10-19T13:59:08.769 DEBUG:teuthology.parallel:result is None 2023-10-19T13:59:08.948 INFO:teuthology.lock.ops:Checking smithi138.front.sepia.ceph.com 2023-10-19T13:59:08.949 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-19T13:59:08.974 INFO:teuthology.lock.ops:Checking smithi190.front.sepia.ceph.com 2023-10-19T13:59:08.974 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-19T13:59:09.000 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: set os type and version on server 2023-10-19T13:59:09.000 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2023-10-19T13:59:09.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:59:09.066 DEBUG:teuthology.orchestra.run.smithi138:> true 2023-10-19T13:59:09.132 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2023-10-19T13:59:09.132 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2023-10-19T13:59:09.189 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2023-10-19T13:59:09.189 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2023-10-19T13:59:09.247 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2023-10-19T13:59:09.247 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="8" 2023-10-19T13:59:09.247 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2023-10-19T13:59:09.247 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2023-10-19T13:59:09.248 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="8" 2023-10-19T13:59:09.248 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el8" 2023-10-19T13:59:09.248 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-19T13:59:09.248 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2023-10-19T13:59:09.248 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-19T13:59:09.249 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2023-10-19T13:59:09.249 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-19T13:59:09.249 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-19T13:59:09.249 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-19T13:59:09.250 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2023-10-19T13:59:09.274 INFO:teuthology.lock.ops:Updating [smithi190.front.sepia.ceph.com]: set os type and version on server 2023-10-19T13:59:09.275 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-10-19T13:59:09.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-19T13:59:09.345 DEBUG:teuthology.orchestra.run.smithi190:> true 2023-10-19T13:59:09.481 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-10-19T13:59:09.482 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2023-10-19T13:59:09.540 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2023-10-19T13:59:09.540 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2023-10-19T13:59:09.598 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream" 2023-10-19T13:59:09.599 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="8" 2023-10-19T13:59:09.599 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2023-10-19T13:59:09.599 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2023-10-19T13:59:09.599 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="8" 2023-10-19T13:59:09.600 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el8" 2023-10-19T13:59:09.600 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-19T13:59:09.600 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2023-10-19T13:59:09.600 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-19T13:59:09.600 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2023-10-19T13:59:09.601 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-19T13:59:09.601 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-19T13:59:09.601 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-19T13:59:09.601 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2023-10-19T13:59:09.624 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T13:59:09.716 INFO:teuthology.dispatcher.supervisor:Running job 7432340 2023-10-19T13:59:09.717 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/virtualenv/bin/teuthology -v --owner scheduled_dparmar@teuthology --archive /home/teuthworker/archive/dparmar-2023-10-19_13:06:44-fs:upgrade-wip-vshankar-testing-20231018.065603-distro-default-smithi/7432340 --name dparmar-2023-10-19_13:06:44-fs:upgrade-wip-vshankar-testing-20231018.065603-distro-default-smithi --description fs:upgrade/upgraded_client/{bluestore-bitmap branch/pacific centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-upgrade 3-workload/stress_tests/{0-client-upgrade 1-tests/blogbench}}} -- /home/teuthworker/archive/dparmar-2023-10-19_13:06:44-fs:upgrade-wip-vshankar-testing-20231018.065603-distro-default-smithi/7432340/orig.config.yaml 2023-10-19T13:59:09.719 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/dparmar-2023-10-19_13:06:44-fs:upgrade-wip-vshankar-testing-20231018.065603-distro-default-smithi/7432340 2023-10-19T13:59:09.720 INFO:teuthology.dispatcher.supervisor:Job PID: 3284841 2023-10-19T13:59:09.720 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-10-19T14:01:09.818 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T14:03:10.003 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T14:05:10.176 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T14:07:10.347 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T14:09:10.516 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T14:11:10.687 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T14:13:10.854 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T14:15:11.067 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T14:17:11.220 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T14:19:11.388 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T14:21:11.501 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T14:23:11.663 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T14:25:11.827 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T14:27:12.004 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T14:29:12.167 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-19T14:29:12.235 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-10-19T14:29:12.419 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-10-19T14:29:12.419 INFO:teuthology.nuke:Checking targets against current locks 2023-10-19T14:29:12.460 DEBUG:teuthology.nuke:shortname: smithi138 2023-10-19T14:29:12.460 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-19T14:29:12.485 DEBUG:teuthology.nuke:shortname: smithi190 2023-10-19T14:29:12.486 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-19T14:29:12.490 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-10-19_13:06:44-fs:upgrade-wip-vshankar-testing-20231018.065603-distro-default-smithi/7432340', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-19 13:53:55.945246', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNzQZahlhbM9xZSaVAITu1ZV+pSudKbbPi5oe29u9ivToMesfXp2Ok2HulFvBzB68v78PaYbvPiUOq5C7hXbjJE='} 2023-10-19T14:29:12.510 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-10-19_13:06:44-fs:upgrade-wip-vshankar-testing-20231018.065603-distro-default-smithi/7432340', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-19 13:53:55.943028', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJZAdPNUlsi45/RVzg+YSOHAnGkQj6jVqhh92dlFrx96GPlYfQaVTQV3ZrOqPU4r5tTC4lQpIpyl1S/5HCOL2nE='} 2023-10-19T14:29:12.542 INFO:teuthology.orchestra.console.smithi138:Power off 2023-10-19T14:29:12.543 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-19T14:29:12.561 INFO:teuthology.orchestra.console.smithi190:Power off 2023-10-19T14:29:12.562 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-19T14:29:12.570 DEBUG:teuthology.orchestra.console.smithi138:power off output: Chassis Power Control: Down/Off 2023-10-19T14:29:12.571 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-19T14:29:12.583 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off 2023-10-19T14:29:12.583 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-19T14:29:12.591 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2023-10-19T14:29:12.607 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2023-10-19T14:29:12.692 INFO:teuthology.orchestra.console.smithi138:Power off completed 2023-10-19T14:29:12.709 INFO:teuthology.orchestra.console.smithi190:Power off completed 2023-10-19T14:29:12.833 INFO:teuthology.lock.ops:unlocked: smithi138.front.sepia.ceph.com 2023-10-19T14:29:12.850 INFO:teuthology.lock.ops:unlocked: smithi190.front.sepia.ceph.com