2023-10-13T20:13:43.244 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-13T20:13:43.246 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-13T20:13:43.391 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1470d8a670>, signals=[15]) 2023-10-13T20:13:43.393 INFO:teuthology.lock.ops:Start node 'smithi029.front.sepia.ceph.com' reimaging 2023-10-13T20:13:43.393 INFO:teuthology.lock.ops:Updating [smithi029.front.sepia.ceph.com]: reset os type and version on server 2023-10-13T20:13:43.394 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2023-10-13T20:13:43.416 INFO:teuthology.lock.ops:Start node 'smithi173.front.sepia.ceph.com' reimaging 2023-10-13T20:13:43.416 INFO:teuthology.lock.ops:Updating [smithi173.front.sepia.ceph.com]: reset os type and version on server 2023-10-13T20:13:43.417 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2023-10-13T20:13:43.418 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /host {"name": "smithi029"} 2023-10-13T20:13:43.439 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /host {"name": "smithi173"} 2023-10-13T20:13:43.493 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-10-13T20:13:43.504 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-10-13T20:13:43.540 DEBUG:teuthology.provision.fog.smithi029:Requesting image smithi_centos_9.stream (ID 10) 2023-10-13T20:13:43.540 DEBUG:teuthology.provision.fog.smithi029:FOG request: PUT /host/134 {"imageID": 10} 2023-10-13T20:13:43.550 DEBUG:teuthology.provision.fog.smithi173:Requesting image smithi_centos_9.stream (ID 10) 2023-10-13T20:13:43.550 DEBUG:teuthology.provision.fog.smithi173:FOG request: PUT /host/278 {"imageID": 10} 2023-10-13T20:13:43.597 INFO:teuthology.provision.fog.smithi029:Scheduling deploy of centos 9.stream 2023-10-13T20:13:43.597 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-10-13T20:13:43.607 INFO:teuthology.provision.fog.smithi173:Scheduling deploy of centos 9.stream 2023-10-13T20:13:43.607 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-10-13T20:13:43.635 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /tasktype {"name": "deploy"} 2023-10-13T20:13:43.646 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /tasktype {"name": "deploy"} 2023-10-13T20:13:43.667 DEBUG:teuthology.provision.fog.smithi029:FOG request: POST /host/134/task {"taskTypeID": 1} 2023-10-13T20:13:43.692 DEBUG:teuthology.provision.fog.smithi173:FOG request: POST /host/278/task {"taskTypeID": 1} 2023-10-13T20:13:43.723 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-10-13T20:13:43.749 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-10-13T20:13:43.800 INFO:teuthology.orchestra.console.smithi029:Power off 2023-10-13T20:13:43.801 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-13T20:13:43.818 INFO:teuthology.orchestra.console.smithi173:Power off 2023-10-13T20:13:43.818 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-13T20:13:43.832 DEBUG:teuthology.orchestra.console.smithi029:power off output: Chassis Power Control: Down/Off 2023-10-13T20:13:43.832 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-13T20:13:43.845 DEBUG:teuthology.orchestra.console.smithi173:power off output: Chassis Power Control: Down/Off 2023-10-13T20:13:43.845 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-13T20:13:43.855 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is on 2023-10-13T20:13:43.868 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2023-10-13T20:13:47.860 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-13T20:13:47.869 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-13T20:13:47.970 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is off 2023-10-13T20:13:47.977 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2023-10-13T20:13:48.072 INFO:teuthology.orchestra.console.smithi029:Power off completed 2023-10-13T20:13:48.172 INFO:teuthology.orchestra.console.smithi029:Power on 2023-10-13T20:13:48.173 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-13T20:13:48.193 DEBUG:teuthology.orchestra.console.smithi029:power on output: Chassis Power Control: Up/On 2023-10-13T20:13:48.194 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-13T20:13:48.215 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is off 2023-10-13T20:13:51.982 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-13T20:13:52.091 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2023-10-13T20:13:52.192 INFO:teuthology.orchestra.console.smithi173:Power off completed 2023-10-13T20:13:52.216 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-13T20:13:52.293 INFO:teuthology.orchestra.console.smithi173:Power on 2023-10-13T20:13:52.294 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-13T20:13:52.316 DEBUG:teuthology.orchestra.console.smithi173:power on output: Chassis Power Control: Up/On 2023-10-13T20:13:52.317 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-13T20:13:52.325 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is on 2023-10-13T20:13:52.339 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2023-10-13T20:13:52.427 INFO:teuthology.orchestra.console.smithi029:Power on completed 2023-10-13T20:13:52.527 INFO:teuthology.provision.fog.smithi029:Waiting for deploy to finish 2023-10-13T20:13:52.528 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-10-13T20:13:56.342 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-13T20:13:56.451 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2023-10-13T20:14:00.454 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-13T20:14:00.563 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2023-10-13T20:14:00.664 INFO:teuthology.orchestra.console.smithi173:Power on completed 2023-10-13T20:14:00.765 INFO:teuthology.provision.fog.smithi173:Waiting for deploy to finish 2023-10-13T20:14:00.765 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-10-13T20:14:07.603 DEBUG:teuthology.provision.fog.smithi029:FOG request: GET /task/active None 2023-10-13T20:14:07.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:14:15.842 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-10-13T20:14:26.170 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-10-13T20:14:30.902 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-10-13T20:14:32.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:14:35.230 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-10-13T20:14:41.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:14:44.317 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-10-13T20:14:45.968 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-10-13T20:14:50.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:14:53.402 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-10-13T20:14:59.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:15:01.032 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-10-13T20:15:02.460 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-10-13T20:15:08.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:15:16.106 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-10-13T20:15:31.186 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-10-13T20:15:40.018 ERROR:teuthology.orchestra.connection:Error authenticating with smithi029.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-10-13T20:15:44.137 ERROR:teuthology.orchestra.connection:Error authenticating with smithi029.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-10-13T20:15:46.242 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-10-13T20:15:51.220 ERROR:teuthology.orchestra.connection:Error authenticating with smithi029.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-10-13T20:16:01.297 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-10-13T20:16:01.301 ERROR:teuthology.orchestra.connection:Error authenticating with smithi029.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-10-13T20:16:14.420 ERROR:teuthology.orchestra.connection:Error authenticating with smithi029.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-10-13T20:16:16.348 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-10-13T20:16:30.539 ERROR:teuthology.orchestra.connection:Error authenticating with smithi029.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-10-13T20:16:31.401 DEBUG:teuthology.provision.fog.smithi173:FOG request: GET /task/active None 2023-10-13T20:16:31.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:16:49.658 ERROR:teuthology.orchestra.connection:Error authenticating with smithi029.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2023-10-13T20:16:49.786 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2023-10-13T20:16:55.788 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:16:58.842 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2023-10-13T20:17:04.844 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:17:07.898 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2023-10-13T20:17:13.900 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:17:16.954 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2023-10-13T20:17:21.786 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-10-13T20:17:22.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:17:26.010 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2023-10-13T20:17:27.788 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:17:30.842 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-10-13T20:17:32.012 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:17:36.844 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:17:39.898 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-10-13T20:17:45.900 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:17:46.042 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-10-13T20:17:52.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:17:55.102 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-10-13T20:18:01.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:18:32.014 WARNING:teuthology.provision.fog.smithi173:timed out 2023-10-13T20:18:38.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:18:38.266 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-13T20:18:38.266 DEBUG:teuthology.orchestra.run.smithi173:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-13T20:18:38.580 INFO:teuthology.provision.fog.smithi173:Node is ready 2023-10-13T20:18:38.580 DEBUG:teuthology.orchestra.run.smithi173:> hostname 2023-10-13T20:18:38.595 INFO:teuthology.orchestra.run.smithi173.stdout:smithi173.front.sepia.ceph.com 2023-10-13T20:18:38.596 DEBUG:teuthology.orchestra.run.smithi173:> grep smithi173.front.sepia.ceph.com /etc/hosts 2023-10-13T20:18:38.652 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.15.173 smithi173.front.sepia.ceph.com smithi173 2023-10-13T20:18:38.652 DEBUG:teuthology.orchestra.run.smithi173:> sudo hostname smithi173 2023-10-13T20:18:38.764 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/smithi173.front.sepia.ceph.com/smithi173/g' /etc/hosts 2023-10-13T20:18:38.792 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/172.21.15.173/172.21.15.173/g' /etc/hosts 2023-10-13T20:18:38.859 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/smithi173.front.sepia.ceph.com/smithi173/g' /etc/hostname 2023-10-13T20:18:38.926 DEBUG:teuthology.orchestra.run.smithi173:> sudo hostname smithi173 2023-10-13T20:18:38.992 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2023-10-13T20:18:39.046 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2023-10-13T20:18:39.047 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="9" 2023-10-13T20:18:39.047 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2023-10-13T20:18:39.047 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2023-10-13T20:18:39.047 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="9" 2023-10-13T20:18:39.048 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el9" 2023-10-13T20:18:39.048 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-13T20:18:39.048 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2023-10-13T20:18:39.048 INFO:teuthology.orchestra.run.smithi173.stdout:LOGO="fedora-logo-icon" 2023-10-13T20:18:39.048 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-13T20:18:39.048 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2023-10-13T20:18:39.049 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-13T20:18:39.049 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-13T20:18:39.049 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-13T20:18:39.050 INFO:teuthology.provision.fog.smithi173:Deploy complete! 2023-10-13T20:18:39.077 DEBUG:teuthology.parallel:result is None 2023-10-13T20:19:01.108 WARNING:teuthology.provision.fog.smithi029:timed out 2023-10-13T20:19:07.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:19:07.209 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-13T20:19:07.210 DEBUG:teuthology.orchestra.run.smithi029:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-13T20:19:07.287 INFO:teuthology.provision.fog.smithi029:Node is ready 2023-10-13T20:19:07.287 DEBUG:teuthology.orchestra.run.smithi029:> hostname 2023-10-13T20:19:07.345 INFO:teuthology.orchestra.run.smithi029.stdout:smithi029 2023-10-13T20:19:07.345 DEBUG:teuthology.orchestra.run.smithi029:> grep smithi029 /etc/hosts 2023-10-13T20:19:07.402 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29 smithi029 smithi029 2023-10-13T20:19:07.402 DEBUG:teuthology.orchestra.run.smithi029:> sudo hostname smithi029 2023-10-13T20:19:07.476 DEBUG:teuthology.orchestra.run.smithi029:> sudo sed -i -e 's/smithi029/smithi029/g' /etc/hosts 2023-10-13T20:19:07.550 DEBUG:teuthology.orchestra.run.smithi029:> sudo sed -i -e 's/172.21.15.29/172.21.15.29/g' /etc/hosts 2023-10-13T20:19:07.623 DEBUG:teuthology.orchestra.run.smithi029:> sudo sed -i -e 's/smithi029/smithi029/g' /etc/hostname 2023-10-13T20:19:07.696 DEBUG:teuthology.orchestra.run.smithi029:> sudo hostname smithi029 2023-10-13T20:19:07.768 DEBUG:teuthology.orchestra.run.smithi029:> cat /etc/os-release 2023-10-13T20:19:07.824 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="CentOS Stream" 2023-10-13T20:19:07.824 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="9" 2023-10-13T20:19:07.824 INFO:teuthology.orchestra.run.smithi029.stdout:ID="centos" 2023-10-13T20:19:07.824 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="rhel fedora" 2023-10-13T20:19:07.825 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="9" 2023-10-13T20:19:07.825 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el9" 2023-10-13T20:19:07.825 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-13T20:19:07.825 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31" 2023-10-13T20:19:07.825 INFO:teuthology.orchestra.run.smithi029.stdout:LOGO="fedora-logo-icon" 2023-10-13T20:19:07.826 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-13T20:19:07.826 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://centos.org/" 2023-10-13T20:19:07.826 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-13T20:19:07.826 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-13T20:19:07.827 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-13T20:19:07.827 INFO:teuthology.provision.fog.smithi029:Deploy complete! 2023-10-13T20:19:07.828 DEBUG:teuthology.parallel:result is None 2023-10-13T20:19:08.023 INFO:teuthology.lock.ops:Checking smithi029.front.sepia.ceph.com 2023-10-13T20:19:08.024 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-13T20:19:08.042 INFO:teuthology.lock.ops:Checking smithi173.front.sepia.ceph.com 2023-10-13T20:19:08.043 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-13T20:19:08.060 INFO:teuthology.lock.ops:Updating [smithi029.front.sepia.ceph.com]: set os type and version on server 2023-10-13T20:19:08.060 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2023-10-13T20:19:08.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:19:08.154 DEBUG:teuthology.orchestra.run.smithi029:> true 2023-10-13T20:19:08.224 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi029.front.sepia.ceph.com' 2023-10-13T20:19:08.225 DEBUG:teuthology.orchestra.run.smithi029:> uname -m 2023-10-13T20:19:08.279 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64 2023-10-13T20:19:08.280 DEBUG:teuthology.orchestra.run.smithi029:> cat /etc/os-release 2023-10-13T20:19:08.334 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="CentOS Stream" 2023-10-13T20:19:08.335 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="9" 2023-10-13T20:19:08.335 INFO:teuthology.orchestra.run.smithi029.stdout:ID="centos" 2023-10-13T20:19:08.335 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="rhel fedora" 2023-10-13T20:19:08.335 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="9" 2023-10-13T20:19:08.336 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el9" 2023-10-13T20:19:08.336 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-13T20:19:08.336 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31" 2023-10-13T20:19:08.336 INFO:teuthology.orchestra.run.smithi029.stdout:LOGO="fedora-logo-icon" 2023-10-13T20:19:08.336 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-13T20:19:08.337 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://centos.org/" 2023-10-13T20:19:08.337 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-13T20:19:08.337 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-13T20:19:08.337 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-13T20:19:08.338 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2023-10-13T20:19:08.359 INFO:teuthology.lock.ops:Updating [smithi173.front.sepia.ceph.com]: set os type and version on server 2023-10-13T20:19:08.360 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2023-10-13T20:19:08.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T20:19:08.444 DEBUG:teuthology.orchestra.run.smithi173:> true 2023-10-13T20:19:08.612 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2023-10-13T20:19:08.613 DEBUG:teuthology.orchestra.run.smithi173:> uname -m 2023-10-13T20:19:08.667 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2023-10-13T20:19:08.667 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2023-10-13T20:19:08.721 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2023-10-13T20:19:08.722 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="9" 2023-10-13T20:19:08.722 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2023-10-13T20:19:08.722 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2023-10-13T20:19:08.722 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="9" 2023-10-13T20:19:08.722 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el9" 2023-10-13T20:19:08.723 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-13T20:19:08.723 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2023-10-13T20:19:08.723 INFO:teuthology.orchestra.run.smithi173.stdout:LOGO="fedora-logo-icon" 2023-10-13T20:19:08.723 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-13T20:19:08.723 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2023-10-13T20:19:08.724 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-13T20:19:08.724 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-13T20:19:08.724 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-13T20:19:08.724 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2023-10-13T20:19:08.742 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-13T20:19:08.831 INFO:teuthology.dispatcher.supervisor:Running job 7424455 2023-10-13T20:19:08.831 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-10-13_14:50:14-rados-wip-neorados-learning-from-experience-distro-default-smithi/7424455 --name yuriw-2023-10-13_14:50:14-rados-wip-neorados-learning-from-experience-distro-default-smithi --description rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-snaps-few-objects-overwrites} -- /home/teuthworker/archive/yuriw-2023-10-13_14:50:14-rados-wip-neorados-learning-from-experience-distro-default-smithi/7424455/orig.config.yaml 2023-10-13T20:19:08.834 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-10-13_14:50:14-rados-wip-neorados-learning-from-experience-distro-default-smithi/7424455 2023-10-13T20:19:08.835 INFO:teuthology.dispatcher.supervisor:Job PID: 3260685 2023-10-13T20:19:08.835 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-10-13T20:21:08.836 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-13T20:21:08.905 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-10-13T20:21:09.008 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi029 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-10-13_14:50:14-rados-wip-neorados-learning-from-experience-distro-default-smithi/7424460 2023-10-13T20:21:09.008 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-10-13T20:21:09.009 INFO:teuthology.nuke:Checking targets against current locks 2023-10-13T20:21:09.060 DEBUG:teuthology.nuke:shortname: smithi029 2023-10-13T20:21:09.061 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-13T20:21:09.082 DEBUG:teuthology.nuke:shortname: smithi173 2023-10-13T20:21:09.083 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-13T20:21:09.086 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi029.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-13_14:50:14-rados-wip-neorados-learning-from-experience-distro-default-smithi/7424460', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-13 20:19:24.812505', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBATsoG7vKNX4q6uR0Edf0HkXBoL/fB6sO9endDJJSkW9WDTOeosISgEVEr8xb0cZlz+ouGqVZdS66qCBRoudPgA='} 2023-10-13T20:21:09.108 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-13_14:50:14-rados-wip-neorados-learning-from-experience-distro-default-smithi/7424455', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-13 20:13:42.425786', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNkP+vRvxxdUcy7mk4S7WcyiXL6t8FCwjU5i+rHM5y6RfNLV//ebxN/FwKXWvW7EsfeJGnF91dk9wbhmP3UME4M='} 2023-10-13T20:21:09.143 INFO:teuthology.orchestra.console.smithi029:Power off 2023-10-13T20:21:09.143 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-13T20:21:09.164 INFO:teuthology.orchestra.console.smithi173:Power off 2023-10-13T20:21:09.164 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-13T20:21:09.173 DEBUG:teuthology.orchestra.console.smithi029:power off output: Chassis Power Control: Down/Off 2023-10-13T20:21:09.173 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-13T20:21:09.184 DEBUG:teuthology.orchestra.console.smithi173:power off output: Chassis Power Control: Down/Off 2023-10-13T20:21:09.185 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-13T20:21:09.204 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is on 2023-10-13T20:21:09.206 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2023-10-13T20:21:09.307 INFO:teuthology.orchestra.console.smithi173:Power off completed 2023-10-13T20:21:09.454 INFO:teuthology.lock.ops:unlocked: smithi173.front.sepia.ceph.com 2023-10-13T20:21:13.205 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-13T20:21:13.315 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is on 2023-10-13T20:21:17.316 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-13T20:21:17.425 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is off 2023-10-13T20:21:17.527 INFO:teuthology.orchestra.console.smithi029:Power off completed 2023-10-13T20:21:17.686 INFO:teuthology.lock.ops:unlocked: smithi029.front.sepia.ceph.com