2023-11-15T06:53:38.475 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-15T06:53:38.477 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T06:53:38.661 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa7721be670>, signals=[15]) 2023-11-15T06:53:38.663 INFO:teuthology.lock.ops:Start node 'smithi124.front.sepia.ceph.com' reimaging 2023-11-15T06:53:38.665 INFO:teuthology.lock.ops:Updating [smithi124.front.sepia.ceph.com]: reset os type and version on server 2023-11-15T06:53:38.665 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2023-11-15T06:53:38.697 INFO:teuthology.lock.ops:Start node 'smithi186.front.sepia.ceph.com' reimaging 2023-11-15T06:53:38.698 INFO:teuthology.lock.ops:Updating [smithi186.front.sepia.ceph.com]: reset os type and version on server 2023-11-15T06:53:38.699 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2023-11-15T06:53:38.701 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /host {"name": "smithi124"} 2023-11-15T06:53:38.739 DEBUG:teuthology.provision.fog.smithi186:FOG request: GET /host {"name": "smithi186"} 2023-11-15T06:53:38.812 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-11-15T06:53:38.839 DEBUG:teuthology.provision.fog.smithi186:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-11-15T06:53:38.886 DEBUG:teuthology.provision.fog.smithi186:Requesting image smithi_centos_8.stream (ID 3) 2023-11-15T06:53:38.886 DEBUG:teuthology.provision.fog.smithi186:FOG request: PUT /host/291 {"imageID": 3} 2023-11-15T06:53:38.898 DEBUG:teuthology.provision.fog.smithi124:Requesting image smithi_centos_8.stream (ID 3) 2023-11-15T06:53:38.898 DEBUG:teuthology.provision.fog.smithi124:FOG request: PUT /host/229 {"imageID": 3} 2023-11-15T06:53:38.970 INFO:teuthology.provision.fog.smithi186:Scheduling deploy of centos 8.stream 2023-11-15T06:53:38.971 DEBUG:teuthology.provision.fog.smithi186:FOG request: GET /task/active None 2023-11-15T06:53:39.001 INFO:teuthology.provision.fog.smithi124:Scheduling deploy of centos 8.stream 2023-11-15T06:53:39.002 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-15T06:53:39.029 DEBUG:teuthology.provision.fog.smithi186:FOG request: GET /tasktype {"name": "deploy"} 2023-11-15T06:53:39.068 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /tasktype {"name": "deploy"} 2023-11-15T06:53:39.079 DEBUG:teuthology.provision.fog.smithi186:FOG request: POST /host/291/task {"taskTypeID": 1} 2023-11-15T06:53:39.119 DEBUG:teuthology.provision.fog.smithi124:FOG request: POST /host/229/task {"taskTypeID": 1} 2023-11-15T06:53:39.239 DEBUG:teuthology.provision.fog.smithi186:FOG request: GET /task/active None 2023-11-15T06:53:39.256 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-15T06:53:39.355 INFO:teuthology.orchestra.console.smithi186:Power off 2023-11-15T06:53:39.356 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-15T06:53:39.379 INFO:teuthology.orchestra.console.smithi124:Power off 2023-11-15T06:53:39.380 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-15T06:53:39.408 DEBUG:teuthology.orchestra.console.smithi124:power off output: Chassis Power Control: Down/Off 2023-11-15T06:53:39.408 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T06:53:39.425 DEBUG:teuthology.orchestra.console.smithi186:power off output: Chassis Power Control: Down/Off 2023-11-15T06:53:39.426 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T06:53:39.445 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on 2023-11-15T06:53:39.458 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-11-15T06:53:43.450 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T06:53:43.460 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T06:53:43.560 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on 2023-11-15T06:53:43.570 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-11-15T06:53:47.564 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T06:53:47.572 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T06:53:47.675 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2023-11-15T06:53:47.681 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2023-11-15T06:53:47.777 INFO:teuthology.orchestra.console.smithi124:Power off completed 2023-11-15T06:53:47.782 INFO:teuthology.orchestra.console.smithi186:Power off completed 2023-11-15T06:53:47.878 INFO:teuthology.orchestra.console.smithi124:Power on 2023-11-15T06:53:47.878 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-15T06:53:47.888 INFO:teuthology.orchestra.console.smithi186:Power on 2023-11-15T06:53:47.889 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-15T06:53:47.904 DEBUG:teuthology.orchestra.console.smithi124:power on output: Chassis Power Control: Up/On 2023-11-15T06:53:47.905 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T06:53:47.914 DEBUG:teuthology.orchestra.console.smithi186:power on output: Chassis Power Control: Up/On 2023-11-15T06:53:47.915 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T06:53:47.928 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2023-11-15T06:53:47.934 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2023-11-15T06:53:51.930 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T06:53:51.941 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T06:53:52.043 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2023-11-15T06:53:52.052 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2023-11-15T06:53:56.046 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T06:53:56.054 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T06:53:56.156 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on 2023-11-15T06:53:56.162 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-11-15T06:53:56.258 INFO:teuthology.orchestra.console.smithi124:Power on completed 2023-11-15T06:53:56.264 INFO:teuthology.orchestra.console.smithi186:Power on completed 2023-11-15T06:53:56.359 INFO:teuthology.provision.fog.smithi124:Waiting for deploy to finish 2023-11-15T06:53:56.360 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-15T06:53:56.365 INFO:teuthology.provision.fog.smithi186:Waiting for deploy to finish 2023-11-15T06:53:56.365 DEBUG:teuthology.provision.fog.smithi186:FOG request: GET /task/active None 2023-11-15T06:54:11.434 DEBUG:teuthology.provision.fog.smithi186:FOG request: GET /task/active None 2023-11-15T06:54:11.436 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-15T06:54:26.510 DEBUG:teuthology.provision.fog.smithi186:FOG request: GET /task/active None 2023-11-15T06:54:26.512 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-15T06:54:41.609 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-15T06:54:41.611 DEBUG:teuthology.provision.fog.smithi186:FOG request: GET /task/active None 2023-11-15T06:54:56.690 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-15T06:54:56.691 DEBUG:teuthology.provision.fog.smithi186:FOG request: GET /task/active None 2023-11-15T06:55:11.782 DEBUG:teuthology.provision.fog.smithi186:FOG request: GET /task/active None 2023-11-15T06:55:11.784 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-15T06:55:26.874 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-15T06:55:26.876 DEBUG:teuthology.provision.fog.smithi186:FOG request: GET /task/active None 2023-11-15T06:55:41.975 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-15T06:55:41.977 DEBUG:teuthology.provision.fog.smithi186:FOG request: GET /task/active None 2023-11-15T06:55:57.074 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-15T06:55:57.076 DEBUG:teuthology.provision.fog.smithi186:FOG request: GET /task/active None 2023-11-15T06:56:12.182 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-15T06:56:12.184 DEBUG:teuthology.provision.fog.smithi186:FOG request: GET /task/active None 2023-11-15T06:56:27.288 DEBUG:teuthology.provision.fog.smithi186:FOG request: GET /task/active None 2023-11-15T06:56:27.289 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-15T06:56:42.398 DEBUG:teuthology.provision.fog.smithi124:FOG request: GET /task/active None 2023-11-15T06:56:42.399 DEBUG:teuthology.provision.fog.smithi186:FOG request: GET /task/active None 2023-11-15T06:56:42.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:56:42.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:56:45.562 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-11-15T06:56:45.563 WARNING:teuthology.provision.fog.smithi186:[Errno None] Unable to connect to port 22 on 172.21.15.186 2023-11-15T06:56:51.566 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:56:51.569 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:56:54.618 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-11-15T06:56:54.654 WARNING:teuthology.provision.fog.smithi186:[Errno None] Unable to connect to port 22 on 172.21.15.186 2023-11-15T06:57:00.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:57:00.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:57:03.678 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-11-15T06:57:03.706 WARNING:teuthology.provision.fog.smithi186:[Errno None] Unable to connect to port 22 on 172.21.15.186 2023-11-15T06:57:09.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:57:09.708 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:57:12.762 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-11-15T06:57:12.763 WARNING:teuthology.provision.fog.smithi186:[Errno None] Unable to connect to port 22 on 172.21.15.186 2023-11-15T06:57:18.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:57:18.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:57:21.818 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2023-11-15T06:57:21.822 WARNING:teuthology.provision.fog.smithi186:[Errno None] Unable to connect to port 22 on 172.21.15.186 2023-11-15T06:57:27.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:57:27.825 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:57:30.878 WARNING:teuthology.provision.fog.smithi186:[Errno None] Unable to connect to port 22 on 172.21.15.186 2023-11-15T06:57:36.882 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:57:39.962 WARNING:teuthology.provision.fog.smithi186:[Errno None] Unable to connect to port 22 on 172.21.15.186 2023-11-15T06:57:45.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:58:27.866 WARNING:teuthology.provision.fog.smithi124:timed out 2023-11-15T06:58:33.871 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:58:34.099 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-15T06:58:34.100 DEBUG:teuthology.orchestra.run.smithi124:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-15T06:58:34.381 INFO:teuthology.provision.fog.smithi124:Node is ready 2023-11-15T06:58:34.381 DEBUG:teuthology.orchestra.run.smithi124:> hostname 2023-11-15T06:58:34.402 INFO:teuthology.orchestra.run.smithi124.stdout:smithi124.front.sepia.ceph.com 2023-11-15T06:58:34.403 DEBUG:teuthology.orchestra.run.smithi124:> grep smithi124.front.sepia.ceph.com /etc/hosts 2023-11-15T06:58:34.465 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.15.124 smithi124.front.sepia.ceph.com smithi124 2023-11-15T06:58:34.465 DEBUG:teuthology.orchestra.run.smithi124:> sudo hostname smithi124 2023-11-15T06:58:34.564 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i -e 's/smithi124.front.sepia.ceph.com/smithi124/g' /etc/hosts 2023-11-15T06:58:34.598 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i -e 's/172.21.15.124/172.21.15.124/g' /etc/hosts 2023-11-15T06:58:34.672 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i -e 's/smithi124.front.sepia.ceph.com/smithi124/g' /etc/hostname 2023-11-15T06:58:34.742 DEBUG:teuthology.orchestra.run.smithi124:> sudo hostname smithi124 2023-11-15T06:58:34.811 DEBUG:teuthology.orchestra.run.smithi124:> cat /etc/os-release 2023-11-15T06:58:34.871 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="CentOS Stream" 2023-11-15T06:58:34.872 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="8" 2023-11-15T06:58:34.872 INFO:teuthology.orchestra.run.smithi124.stdout:ID="centos" 2023-11-15T06:58:34.872 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE="rhel fedora" 2023-11-15T06:58:34.872 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="8" 2023-11-15T06:58:34.873 INFO:teuthology.orchestra.run.smithi124.stdout:PLATFORM_ID="platform:el8" 2023-11-15T06:58:34.873 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-15T06:58:34.873 INFO:teuthology.orchestra.run.smithi124.stdout:ANSI_COLOR="0;31" 2023-11-15T06:58:34.873 INFO:teuthology.orchestra.run.smithi124.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-15T06:58:34.873 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://centos.org/" 2023-11-15T06:58:34.874 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-15T06:58:34.874 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-15T06:58:34.874 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-15T06:58:34.875 INFO:teuthology.provision.fog.smithi124:Deploy complete! 2023-11-15T06:58:34.876 DEBUG:teuthology.parallel:result is None 2023-11-15T06:58:45.979 WARNING:teuthology.provision.fog.smithi186:timed out 2023-11-15T06:58:51.986 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:58:52.217 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-15T06:58:52.218 DEBUG:teuthology.orchestra.run.smithi186:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-15T06:58:52.462 INFO:teuthology.provision.fog.smithi186:Node is ready 2023-11-15T06:58:52.462 DEBUG:teuthology.orchestra.run.smithi186:> hostname 2023-11-15T06:58:52.482 INFO:teuthology.orchestra.run.smithi186.stdout:smithi186.front.sepia.ceph.com 2023-11-15T06:58:52.482 DEBUG:teuthology.orchestra.run.smithi186:> grep smithi186.front.sepia.ceph.com /etc/hosts 2023-11-15T06:58:52.542 INFO:teuthology.orchestra.run.smithi186.stdout:172.21.15.186 smithi186.front.sepia.ceph.com smithi186 2023-11-15T06:58:52.543 DEBUG:teuthology.orchestra.run.smithi186:> sudo hostname smithi186 2023-11-15T06:58:52.650 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i -e 's/smithi186.front.sepia.ceph.com/smithi186/g' /etc/hosts 2023-11-15T06:58:52.681 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i -e 's/172.21.15.186/172.21.15.186/g' /etc/hosts 2023-11-15T06:58:52.749 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i -e 's/smithi186.front.sepia.ceph.com/smithi186/g' /etc/hostname 2023-11-15T06:58:52.819 DEBUG:teuthology.orchestra.run.smithi186:> sudo hostname smithi186 2023-11-15T06:58:52.889 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2023-11-15T06:58:52.948 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="CentOS Stream" 2023-11-15T06:58:52.948 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="8" 2023-11-15T06:58:52.948 INFO:teuthology.orchestra.run.smithi186.stdout:ID="centos" 2023-11-15T06:58:52.948 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="rhel fedora" 2023-11-15T06:58:52.949 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="8" 2023-11-15T06:58:52.949 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el8" 2023-11-15T06:58:52.949 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-15T06:58:52.949 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2023-11-15T06:58:52.949 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-15T06:58:52.950 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://centos.org/" 2023-11-15T06:58:52.950 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-15T06:58:52.950 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-15T06:58:52.950 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-15T06:58:52.951 INFO:teuthology.provision.fog.smithi186:Deploy complete! 2023-11-15T06:58:52.951 DEBUG:teuthology.parallel:result is None 2023-11-15T06:58:53.106 INFO:teuthology.lock.ops:Checking smithi124.front.sepia.ceph.com 2023-11-15T06:58:53.107 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-15T06:58:53.132 INFO:teuthology.lock.ops:Checking smithi186.front.sepia.ceph.com 2023-11-15T06:58:53.132 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-15T06:58:53.149 INFO:teuthology.lock.ops:Updating [smithi124.front.sepia.ceph.com]: set os type and version on server 2023-11-15T06:58:53.150 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-11-15T06:58:53.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:58:53.219 DEBUG:teuthology.orchestra.run.smithi124:> true 2023-11-15T06:58:53.362 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi124.front.sepia.ceph.com' 2023-11-15T06:58:53.363 DEBUG:teuthology.orchestra.run.smithi124:> uname -m 2023-11-15T06:58:53.422 INFO:teuthology.orchestra.run.smithi124.stdout:x86_64 2023-11-15T06:58:53.422 DEBUG:teuthology.orchestra.run.smithi124:> cat /etc/os-release 2023-11-15T06:58:53.481 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="CentOS Stream" 2023-11-15T06:58:53.481 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="8" 2023-11-15T06:58:53.481 INFO:teuthology.orchestra.run.smithi124.stdout:ID="centos" 2023-11-15T06:58:53.482 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE="rhel fedora" 2023-11-15T06:58:53.482 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="8" 2023-11-15T06:58:53.482 INFO:teuthology.orchestra.run.smithi124.stdout:PLATFORM_ID="platform:el8" 2023-11-15T06:58:53.482 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-15T06:58:53.482 INFO:teuthology.orchestra.run.smithi124.stdout:ANSI_COLOR="0;31" 2023-11-15T06:58:53.483 INFO:teuthology.orchestra.run.smithi124.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-15T06:58:53.483 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://centos.org/" 2023-11-15T06:58:53.483 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-15T06:58:53.483 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-15T06:58:53.483 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-15T06:58:53.484 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2023-11-15T06:58:53.507 INFO:teuthology.lock.ops:Updating [smithi186.front.sepia.ceph.com]: set os type and version on server 2023-11-15T06:58:53.508 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-11-15T06:58:53.508 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-15T06:58:53.575 DEBUG:teuthology.orchestra.run.smithi186:> true 2023-11-15T06:58:53.645 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-11-15T06:58:53.646 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2023-11-15T06:58:53.704 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2023-11-15T06:58:53.705 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2023-11-15T06:58:53.763 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="CentOS Stream" 2023-11-15T06:58:53.764 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="8" 2023-11-15T06:58:53.764 INFO:teuthology.orchestra.run.smithi186.stdout:ID="centos" 2023-11-15T06:58:53.764 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="rhel fedora" 2023-11-15T06:58:53.764 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="8" 2023-11-15T06:58:53.765 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el8" 2023-11-15T06:58:53.765 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="CentOS Stream 8" 2023-11-15T06:58:53.765 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2023-11-15T06:58:53.765 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-11-15T06:58:53.765 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://centos.org/" 2023-11-15T06:58:53.766 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-15T06:58:53.766 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-15T06:58:53.766 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-15T06:58:53.767 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2023-11-15T06:58:53.791 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T06:58:53.884 INFO:teuthology.dispatcher.supervisor:Running job 7458701 2023-11-15T06:58:53.884 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2023-11-15_03:42:13-orch:cephadm-wip-adk-testing-2023-11-14-1702-distro-default-smithi/7458701 --name adking-2023-11-15_03:42:13-orch:cephadm-wip-adk-testing-2023-11-14-1702-distro-default-smithi --description orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/adking-2023-11-15_03:42:13-orch:cephadm-wip-adk-testing-2023-11-14-1702-distro-default-smithi/7458701/orig.config.yaml 2023-11-15T06:58:53.889 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2023-11-15_03:42:13-orch:cephadm-wip-adk-testing-2023-11-14-1702-distro-default-smithi/7458701 2023-11-15T06:58:53.890 INFO:teuthology.dispatcher.supervisor:Job PID: 1783004 2023-11-15T06:58:53.891 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-15T07:00:53.991 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T07:02:54.161 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T07:04:54.331 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T07:06:54.495 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T07:08:54.663 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T07:10:54.831 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T07:12:54.999 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T07:14:55.167 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T07:16:55.335 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T07:18:55.499 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T07:20:55.687 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T07:22:55.856 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T07:24:56.031 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T07:26:56.203 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T07:28:56.367 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-15T07:28:56.437 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-11-15T07:28:56.608 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-15T07:28:56.609 INFO:teuthology.nuke:Checking targets against current locks 2023-11-15T07:28:56.665 DEBUG:teuthology.nuke:shortname: smithi124 2023-11-15T07:28:56.665 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-15T07:28:56.684 DEBUG:teuthology.nuke:shortname: smithi186 2023-11-15T07:28:56.684 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-15T07:28:56.689 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi124.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2023-11-15_03:42:13-orch:cephadm-wip-adk-testing-2023-11-14-1702-distro-default-smithi/7458701', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-15 06:53:37.564591', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOjSfauI6t39piw2RXsygdvRy3Vg9d+uF/FVoGABvxuFkKcTQjSSrJ4LcgUQml4YvWQxM18LO/8qByO6qyYHfNc='} 2023-11-15T07:28:56.703 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2023-11-15_03:42:13-orch:cephadm-wip-adk-testing-2023-11-14-1702-distro-default-smithi/7458701', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-15 06:53:37.566265', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBjWetjEJZWmGaQ0f2BJO814WCnCICn1R+cd44tiEblZyaFr0OBIKYKU262gopur0ebrC25S0PUm8d2NP7Oh9qg='} 2023-11-15T07:28:56.733 INFO:teuthology.orchestra.console.smithi124:Power off 2023-11-15T07:28:56.733 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-15T07:28:56.759 DEBUG:teuthology.orchestra.console.smithi124:power off output: Chassis Power Control: Down/Off 2023-11-15T07:28:56.759 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T07:28:56.782 INFO:teuthology.orchestra.console.smithi186:Power off 2023-11-15T07:28:56.783 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-15T07:28:56.793 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2023-11-15T07:28:56.806 DEBUG:teuthology.orchestra.console.smithi186:power off output: Chassis Power Control: Down/Off 2023-11-15T07:28:56.806 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-15T07:28:56.834 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2023-11-15T07:28:56.894 INFO:teuthology.orchestra.console.smithi124:Power off completed 2023-11-15T07:28:56.936 INFO:teuthology.orchestra.console.smithi186:Power off completed 2023-11-15T07:28:57.045 INFO:teuthology.lock.ops:unlocked: smithi124.front.sepia.ceph.com 2023-11-15T07:28:57.087 INFO:teuthology.lock.ops:unlocked: smithi186.front.sepia.ceph.com