2023-11-12T08:51:19.481 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-12T08:51:19.482 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T08:51:19.604 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4e6804db80>, signals=[15]) 2023-11-12T08:51:19.605 INFO:teuthology.lock.ops:Start node 'smithi077.front.sepia.ceph.com' reimaging 2023-11-12T08:51:19.606 INFO:teuthology.lock.ops:Updating [smithi077.front.sepia.ceph.com]: reset os type and version on server 2023-11-12T08:51:19.606 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2023-11-12T08:51:19.621 INFO:teuthology.lock.ops:Start node 'smithi094.front.sepia.ceph.com' reimaging 2023-11-12T08:51:19.621 INFO:teuthology.lock.ops:Updating [smithi094.front.sepia.ceph.com]: reset os type and version on server 2023-11-12T08:51:19.621 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2023-11-12T08:51:19.623 DEBUG:teuthology.provision.fog.smithi077:FOG request: GET /host {"name": "smithi077"} 2023-11-12T08:51:19.646 DEBUG:teuthology.provision.fog.smithi094:FOG request: GET /host {"name": "smithi094"} 2023-11-12T08:51:19.687 DEBUG:teuthology.provision.fog.smithi077:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-11-12T08:51:19.700 DEBUG:teuthology.provision.fog.smithi094:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-11-12T08:51:19.726 DEBUG:teuthology.provision.fog.smithi077:Requesting image smithi_centos_9.stream (ID 10) 2023-11-12T08:51:19.727 DEBUG:teuthology.provision.fog.smithi077:FOG request: PUT /host/182 {"imageID": 10} 2023-11-12T08:51:19.738 DEBUG:teuthology.provision.fog.smithi094:Requesting image smithi_centos_9.stream (ID 10) 2023-11-12T08:51:19.739 DEBUG:teuthology.provision.fog.smithi094:FOG request: PUT /host/199 {"imageID": 10} 2023-11-12T08:51:19.778 INFO:teuthology.provision.fog.smithi094:Scheduling deploy of centos 9.stream 2023-11-12T08:51:19.779 DEBUG:teuthology.provision.fog.smithi094:FOG request: GET /task/active None 2023-11-12T08:51:19.783 INFO:teuthology.provision.fog.smithi077:Scheduling deploy of centos 9.stream 2023-11-12T08:51:19.783 DEBUG:teuthology.provision.fog.smithi077:FOG request: GET /task/active None 2023-11-12T08:51:19.824 DEBUG:teuthology.provision.fog.smithi094:FOG request: GET /tasktype {"name": "deploy"} 2023-11-12T08:51:19.837 DEBUG:teuthology.provision.fog.smithi077:FOG request: GET /tasktype {"name": "deploy"} 2023-11-12T08:51:19.863 DEBUG:teuthology.provision.fog.smithi094:FOG request: POST /host/199/task {"taskTypeID": 1} 2023-11-12T08:51:19.869 DEBUG:teuthology.provision.fog.smithi077:FOG request: POST /host/182/task {"taskTypeID": 1} 2023-11-12T08:51:19.922 DEBUG:teuthology.provision.fog.smithi094:FOG request: GET /task/active None 2023-11-12T08:51:19.924 DEBUG:teuthology.provision.fog.smithi077:FOG request: GET /task/active None 2023-11-12T08:51:20.006 INFO:teuthology.orchestra.console.smithi077:Power off 2023-11-12T08:51:20.007 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-12T08:51:20.019 INFO:teuthology.orchestra.console.smithi094:Power off 2023-11-12T08:51:20.020 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-12T08:51:20.027 DEBUG:teuthology.orchestra.console.smithi077:power off output: Chassis Power Control: Down/Off 2023-11-12T08:51:20.028 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-12T08:51:20.050 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is on 2023-11-12T08:51:20.057 DEBUG:teuthology.orchestra.console.smithi094:power off output: Chassis Power Control: Down/Off 2023-11-12T08:51:20.057 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-12T08:51:20.079 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2023-11-12T08:51:24.054 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-12T08:51:24.080 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-12T08:51:24.163 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is on 2023-11-12T08:51:24.188 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2023-11-12T08:51:28.168 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-12T08:51:28.189 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-12T08:51:28.277 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is off 2023-11-12T08:51:28.298 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2023-11-12T08:51:28.378 INFO:teuthology.orchestra.console.smithi077:Power off completed 2023-11-12T08:51:28.479 INFO:teuthology.orchestra.console.smithi077:Power on 2023-11-12T08:51:28.479 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-12T08:51:28.499 DEBUG:teuthology.orchestra.console.smithi077:power on output: Chassis Power Control: Up/On 2023-11-12T08:51:28.499 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-12T08:51:28.520 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is off 2023-11-12T08:51:32.303 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-12T08:51:32.414 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is off 2023-11-12T08:51:32.515 INFO:teuthology.orchestra.console.smithi094:Power off completed 2023-11-12T08:51:32.521 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-12T08:51:32.617 INFO:teuthology.orchestra.console.smithi094:Power on 2023-11-12T08:51:32.617 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-12T08:51:32.631 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is off 2023-11-12T08:51:32.640 DEBUG:teuthology.orchestra.console.smithi094:power on output: Chassis Power Control: Up/On 2023-11-12T08:51:32.640 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-12T08:51:32.661 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is off 2023-11-12T08:51:36.636 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-12T08:51:36.662 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-12T08:51:36.773 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2023-11-12T08:51:36.837 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is on 2023-11-12T08:51:36.874 INFO:teuthology.orchestra.console.smithi094:Power on completed 2023-11-12T08:51:36.938 INFO:teuthology.orchestra.console.smithi077:Power on completed 2023-11-12T08:51:36.976 INFO:teuthology.provision.fog.smithi094:Waiting for deploy to finish 2023-11-12T08:51:36.976 DEBUG:teuthology.provision.fog.smithi094:FOG request: GET /task/active None 2023-11-12T08:51:37.038 INFO:teuthology.provision.fog.smithi077:Waiting for deploy to finish 2023-11-12T08:51:37.039 DEBUG:teuthology.provision.fog.smithi077:FOG request: GET /task/active None 2023-11-12T08:51:52.049 DEBUG:teuthology.provision.fog.smithi094:FOG request: GET /task/active None 2023-11-12T08:51:52.097 DEBUG:teuthology.provision.fog.smithi077:FOG request: GET /task/active None 2023-11-12T08:52:07.138 DEBUG:teuthology.provision.fog.smithi094:FOG request: GET /task/active None 2023-11-12T08:52:07.157 DEBUG:teuthology.provision.fog.smithi077:FOG request: GET /task/active None 2023-11-12T08:52:22.216 DEBUG:teuthology.provision.fog.smithi094:FOG request: GET /task/active None 2023-11-12T08:52:22.218 DEBUG:teuthology.provision.fog.smithi077:FOG request: GET /task/active None 2023-11-12T08:52:37.291 DEBUG:teuthology.provision.fog.smithi094:FOG request: GET /task/active None 2023-11-12T08:52:37.293 DEBUG:teuthology.provision.fog.smithi077:FOG request: GET /task/active None 2023-11-12T08:52:52.378 DEBUG:teuthology.provision.fog.smithi094:FOG request: GET /task/active None 2023-11-12T08:52:52.384 DEBUG:teuthology.provision.fog.smithi077:FOG request: GET /task/active None 2023-11-12T08:53:07.464 DEBUG:teuthology.provision.fog.smithi077:FOG request: GET /task/active None 2023-11-12T08:53:07.466 DEBUG:teuthology.provision.fog.smithi094:FOG request: GET /task/active None 2023-11-12T08:53:22.541 DEBUG:teuthology.provision.fog.smithi094:FOG request: GET /task/active None 2023-11-12T08:53:22.543 DEBUG:teuthology.provision.fog.smithi077:FOG request: GET /task/active None 2023-11-12T08:53:37.618 DEBUG:teuthology.provision.fog.smithi094:FOG request: GET /task/active None 2023-11-12T08:53:37.619 DEBUG:teuthology.provision.fog.smithi077:FOG request: GET /task/active None 2023-11-12T08:53:52.697 DEBUG:teuthology.provision.fog.smithi094:FOG request: GET /task/active None 2023-11-12T08:53:52.699 DEBUG:teuthology.provision.fog.smithi077:FOG request: GET /task/active None 2023-11-12T08:54:07.778 DEBUG:teuthology.provision.fog.smithi094:FOG request: GET /task/active None 2023-11-12T08:54:07.780 DEBUG:teuthology.provision.fog.smithi077:FOG request: GET /task/active None 2023-11-12T08:54:07.836 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-12T08:54:07.841 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-12T08:54:08.129 ERROR:teuthology.orchestra.connection:Error authenticating with smithi077.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-11-12T08:54:26.173 WARNING:teuthology.provision.fog.smithi094:[Errno None] Unable to connect to port 22 on 172.21.15.94 2023-11-12T08:54:32.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-12T08:54:35.258 WARNING:teuthology.provision.fog.smithi094:[Errno None] Unable to connect to port 22 on 172.21.15.94 2023-11-12T08:54:41.260 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-12T08:54:44.315 WARNING:teuthology.provision.fog.smithi094:[Errno None] Unable to connect to port 22 on 172.21.15.94 2023-11-12T08:54:50.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-12T08:54:53.374 WARNING:teuthology.provision.fog.smithi094:[Errno None] Unable to connect to port 22 on 172.21.15.94 2023-11-12T08:54:59.376 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-12T08:55:02.459 WARNING:teuthology.provision.fog.smithi094:[Errno None] Unable to connect to port 22 on 172.21.15.94 2023-11-12T08:55:08.462 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-12T08:55:12.135 WARNING:teuthology.provision.fog.smithi077:timed out 2023-11-12T08:55:18.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-12T08:55:36.571 WARNING:teuthology.provision.fog.smithi077:[Errno None] Unable to connect to port 22 on 172.21.15.77 2023-11-12T08:55:42.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-12T08:55:45.659 WARNING:teuthology.provision.fog.smithi077:[Errno None] Unable to connect to port 22 on 172.21.15.77 2023-11-12T08:55:51.660 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-12T08:55:51.917 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-12T08:55:51.918 DEBUG:teuthology.orchestra.run.smithi077:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-12T08:56:08.465 WARNING:teuthology.provision.fog.smithi094:timed out 2023-11-12T08:56:14.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-12T08:56:14.884 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-12T08:56:14.884 DEBUG:teuthology.orchestra.run.smithi094:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-12T08:56:15.233 INFO:teuthology.provision.fog.smithi094:Node is ready 2023-11-12T08:56:15.234 DEBUG:teuthology.orchestra.run.smithi094:> hostname 2023-11-12T08:56:15.248 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094.front.sepia.ceph.com 2023-11-12T08:56:15.249 DEBUG:teuthology.orchestra.run.smithi094:> grep smithi094.front.sepia.ceph.com /etc/hosts 2023-11-12T08:56:15.307 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94 smithi094.front.sepia.ceph.com smithi094 2023-11-12T08:56:15.308 DEBUG:teuthology.orchestra.run.smithi094:> sudo hostname smithi094 2023-11-12T08:56:15.432 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i -e 's/smithi094.front.sepia.ceph.com/smithi094/g' /etc/hosts 2023-11-12T08:56:15.460 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i -e 's/172.21.15.94/172.21.15.94/g' /etc/hosts 2023-11-12T08:56:15.532 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i -e 's/smithi094.front.sepia.ceph.com/smithi094/g' /etc/hostname 2023-11-12T08:56:15.604 DEBUG:teuthology.orchestra.run.smithi094:> sudo hostname smithi094 2023-11-12T08:56:15.673 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2023-11-12T08:56:15.732 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2023-11-12T08:56:15.732 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2023-11-12T08:56:15.733 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2023-11-12T08:56:15.733 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2023-11-12T08:56:15.733 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2023-11-12T08:56:15.733 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2023-11-12T08:56:15.734 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-12T08:56:15.734 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2023-11-12T08:56:15.734 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2023-11-12T08:56:15.734 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-12T08:56:15.735 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2023-11-12T08:56:15.735 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-12T08:56:15.735 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-12T08:56:15.735 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-12T08:56:15.736 INFO:teuthology.provision.fog.smithi094:Deploy complete! 2023-11-12T08:56:15.738 DEBUG:teuthology.parallel:result is None 2023-11-12T08:56:17.242 INFO:teuthology.provision.fog.smithi077:Node is ready 2023-11-12T08:56:17.242 DEBUG:teuthology.orchestra.run.smithi077:> hostname 2023-11-12T08:56:17.261 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077.front.sepia.ceph.com 2023-11-12T08:56:17.262 DEBUG:teuthology.orchestra.run.smithi077:> grep smithi077.front.sepia.ceph.com /etc/hosts 2023-11-12T08:56:17.319 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77 smithi077.front.sepia.ceph.com smithi077 2023-11-12T08:56:17.319 DEBUG:teuthology.orchestra.run.smithi077:> sudo hostname smithi077 2023-11-12T08:56:17.447 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i -e 's/smithi077.front.sepia.ceph.com/smithi077/g' /etc/hosts 2023-11-12T08:56:17.478 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i -e 's/172.21.15.77/172.21.15.77/g' /etc/hosts 2023-11-12T08:56:17.548 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i -e 's/smithi077.front.sepia.ceph.com/smithi077/g' /etc/hostname 2023-11-12T08:56:17.616 DEBUG:teuthology.orchestra.run.smithi077:> sudo hostname smithi077 2023-11-12T08:56:17.683 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2023-11-12T08:56:17.738 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream" 2023-11-12T08:56:17.738 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="9" 2023-11-12T08:56:17.738 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos" 2023-11-12T08:56:17.738 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora" 2023-11-12T08:56:17.739 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="9" 2023-11-12T08:56:17.739 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el9" 2023-11-12T08:56:17.739 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-12T08:56:17.739 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2023-11-12T08:56:17.739 INFO:teuthology.orchestra.run.smithi077.stdout:LOGO="fedora-logo-icon" 2023-11-12T08:56:17.740 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-12T08:56:17.740 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/" 2023-11-12T08:56:17.740 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-12T08:56:17.740 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-12T08:56:17.740 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-12T08:56:17.741 INFO:teuthology.provision.fog.smithi077:Deploy complete! 2023-11-12T08:56:17.741 DEBUG:teuthology.parallel:result is None 2023-11-12T08:56:17.903 INFO:teuthology.lock.ops:Checking smithi077.front.sepia.ceph.com 2023-11-12T08:56:17.904 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-12T08:56:17.929 INFO:teuthology.lock.ops:Checking smithi094.front.sepia.ceph.com 2023-11-12T08:56:17.930 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-12T08:56:17.955 INFO:teuthology.lock.ops:Updating [smithi077.front.sepia.ceph.com]: set os type and version on server 2023-11-12T08:56:17.955 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2023-11-12T08:56:17.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-12T08:56:18.038 DEBUG:teuthology.orchestra.run.smithi077:> true 2023-11-12T08:56:18.104 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2023-11-12T08:56:18.105 DEBUG:teuthology.orchestra.run.smithi077:> uname -m 2023-11-12T08:56:18.158 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2023-11-12T08:56:18.159 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2023-11-12T08:56:18.212 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream" 2023-11-12T08:56:18.213 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="9" 2023-11-12T08:56:18.213 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos" 2023-11-12T08:56:18.213 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora" 2023-11-12T08:56:18.213 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="9" 2023-11-12T08:56:18.214 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el9" 2023-11-12T08:56:18.214 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-12T08:56:18.214 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2023-11-12T08:56:18.214 INFO:teuthology.orchestra.run.smithi077.stdout:LOGO="fedora-logo-icon" 2023-11-12T08:56:18.215 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-12T08:56:18.215 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/" 2023-11-12T08:56:18.215 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-12T08:56:18.215 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-12T08:56:18.215 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-12T08:56:18.216 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2023-11-12T08:56:18.246 INFO:teuthology.lock.ops:Updating [smithi094.front.sepia.ceph.com]: set os type and version on server 2023-11-12T08:56:18.247 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2023-11-12T08:56:18.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-12T08:56:18.333 DEBUG:teuthology.orchestra.run.smithi094:> true 2023-11-12T08:56:18.402 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2023-11-12T08:56:18.402 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2023-11-12T08:56:18.464 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2023-11-12T08:56:18.464 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2023-11-12T08:56:18.523 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2023-11-12T08:56:18.523 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2023-11-12T08:56:18.523 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2023-11-12T08:56:18.523 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2023-11-12T08:56:18.524 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2023-11-12T08:56:18.524 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2023-11-12T08:56:18.524 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-12T08:56:18.524 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2023-11-12T08:56:18.524 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2023-11-12T08:56:18.524 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-12T08:56:18.525 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2023-11-12T08:56:18.525 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-12T08:56:18.525 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-12T08:56:18.525 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-12T08:56:18.526 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2023-11-12T08:56:18.554 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T08:56:18.625 INFO:teuthology.dispatcher.supervisor:Running job 7456026 2023-11-12T08:56:18.625 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_nmordech@teuthology --archive /home/teuthworker/archive/nmordech-2023-11-12_08:50:05-rados:thrash-dedup-main-distro-default-smithi/7456026 --name nmordech-2023-11-12_08:50:05-rados:thrash-dedup-main-distro-default-smithi --description rados:thrash-dedup/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} d-balancer/on mon_election/classic msgr-failures/fastclose msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_latest overrides} thrashers/default thrashosds-health workloads/dedup-io-mixed} -- /home/teuthworker/archive/nmordech-2023-11-12_08:50:05-rados:thrash-dedup-main-distro-default-smithi/7456026/orig.config.yaml 2023-11-12T08:56:18.628 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/nmordech-2023-11-12_08:50:05-rados:thrash-dedup-main-distro-default-smithi/7456026 2023-11-12T08:56:18.628 INFO:teuthology.dispatcher.supervisor:Job PID: 2020621 2023-11-12T08:56:18.628 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-12T08:58:18.630 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:00:18.699 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:02:18.769 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:04:18.832 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:06:18.912 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:08:18.973 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:10:19.037 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:12:19.108 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:14:19.171 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:16:19.243 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:18:19.306 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:20:19.379 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:22:19.450 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:24:19.510 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:26:19.583 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:28:19.645 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:30:19.713 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:32:19.783 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-12T09:32:19.854 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-11-12T09:32:19.960 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-12T09:32:19.960 INFO:teuthology.nuke:Checking targets against current locks 2023-11-12T09:32:20.004 DEBUG:teuthology.nuke:shortname: smithi077 2023-11-12T09:32:20.005 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-12T09:32:20.025 DEBUG:teuthology.nuke:shortname: smithi094 2023-11-12T09:32:20.026 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-12T09:32:20.033 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/nmordech-2023-11-12_08:50:05-rados:thrash-dedup-main-distro-default-smithi/7456026', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-12 08:51:18.728173', 'locked_by': 'scheduled_nmordech@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP9bWE0Mh1DWp2Po4tCORyL7FXkPvThte5gkq7xRHZGTuUEKX5k1eI8dC9A4pZce+i78CEloc5UIWzAInIoPE0w='} 2023-11-12T09:32:20.044 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/nmordech-2023-11-12_08:50:05-rados:thrash-dedup-main-distro-default-smithi/7456026', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-12 08:51:18.726866', 'locked_by': 'scheduled_nmordech@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIqmNuZaBpbqg3B1zFlmX33JrC2nE8/XsoJzjiwtpOXWYIU5ikaHCUw88oX81sZRuPCDeFTDWxX9e3KK9O8DCbc='} 2023-11-12T09:32:20.077 INFO:teuthology.orchestra.console.smithi077:Power off 2023-11-12T09:32:20.078 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-12T09:32:20.092 INFO:teuthology.orchestra.console.smithi094:Power off 2023-11-12T09:32:20.092 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-12T09:32:20.100 DEBUG:teuthology.orchestra.console.smithi077:power off output: Chassis Power Control: Down/Off 2023-11-12T09:32:20.100 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-12T09:32:20.111 DEBUG:teuthology.orchestra.console.smithi094:power off output: Chassis Power Control: Down/Off 2023-11-12T09:32:20.112 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-12T09:32:20.119 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is off 2023-11-12T09:32:20.150 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is off 2023-11-12T09:32:20.220 INFO:teuthology.orchestra.console.smithi077:Power off completed 2023-11-12T09:32:20.251 INFO:teuthology.orchestra.console.smithi094:Power off completed 2023-11-12T09:32:20.371 INFO:teuthology.lock.ops:unlocked: smithi077.front.sepia.ceph.com 2023-11-12T09:32:20.402 INFO:teuthology.lock.ops:unlocked: smithi094.front.sepia.ceph.com