2024-03-02T12:09:27.230 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-03-02T12:09:27.231 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:09:27.404 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f27be4bcc10>, signals=[15]) 2024-03-02T12:09:27.404 INFO:teuthology.lock.ops:Start node 'smithi027.front.sepia.ceph.com' reimaging 2024-03-02T12:09:27.404 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: reset os type and version on server 2024-03-02T12:09:27.404 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-03-02T12:09:27.427 INFO:teuthology.lock.ops:Start node 'smithi165.front.sepia.ceph.com' reimaging 2024-03-02T12:09:27.428 INFO:teuthology.lock.ops:Updating [smithi165.front.sepia.ceph.com]: reset os type and version on server 2024-03-02T12:09:27.428 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-03-02T12:09:27.443 INFO:teuthology.lock.ops:Start node 'smithi173.front.sepia.ceph.com' reimaging 2024-03-02T12:09:27.443 INFO:teuthology.lock.ops:Updating [smithi173.front.sepia.ceph.com]: reset os type and version on server 2024-03-02T12:09:27.444 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2024-03-02T12:09:27.538 DEBUG:teuthology.provision.fog.smithi027:Requesting image smithi_centos_9.stream (ID 10) 2024-03-02T12:09:27.548 DEBUG:teuthology.provision.fog.smithi165:Requesting image smithi_centos_9.stream (ID 10) 2024-03-02T12:09:27.565 DEBUG:teuthology.provision.fog.smithi173:Requesting image smithi_centos_9.stream (ID 10) 2024-03-02T12:09:27.581 INFO:teuthology.provision.fog.smithi027:Scheduling deploy of centos 9.stream 2024-03-02T12:09:27.590 INFO:teuthology.provision.fog.smithi165:Scheduling deploy of centos 9.stream 2024-03-02T12:09:27.613 INFO:teuthology.provision.fog.smithi173:Scheduling deploy of centos 9.stream 2024-03-02T12:09:27.844 INFO:teuthology.orchestra.console.smithi027:Power off 2024-03-02T12:09:27.845 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-02T12:09:27.861 INFO:teuthology.orchestra.console.smithi165:Power off 2024-03-02T12:09:27.861 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-02T12:09:27.871 DEBUG:teuthology.orchestra.console.smithi027:power off output: Chassis Power Control: Down/Off 2024-03-02T12:09:27.871 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-02T12:09:27.881 DEBUG:teuthology.orchestra.console.smithi165:power off output: Chassis Power Control: Down/Off 2024-03-02T12:09:27.881 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-02T12:09:27.894 INFO:teuthology.orchestra.console.smithi173:Power off 2024-03-02T12:09:27.895 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-02T12:09:27.902 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off 2024-03-02T12:09:27.902 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is off 2024-03-02T12:09:27.916 DEBUG:teuthology.orchestra.console.smithi173:power off output: Chassis Power Control: Down/Off 2024-03-02T12:09:27.916 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-02T12:09:27.939 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2024-03-02T12:09:28.003 INFO:teuthology.orchestra.console.smithi027:Power off completed 2024-03-02T12:09:28.003 INFO:teuthology.orchestra.console.smithi165:Power off completed 2024-03-02T12:09:28.040 INFO:teuthology.orchestra.console.smithi173:Power off completed 2024-03-02T12:09:28.104 INFO:teuthology.orchestra.console.smithi027:Power on 2024-03-02T12:09:28.105 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-02T12:09:28.112 INFO:teuthology.orchestra.console.smithi165:Power on 2024-03-02T12:09:28.112 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-02T12:09:28.129 DEBUG:teuthology.orchestra.console.smithi027:power on output: Chassis Power Control: Up/On 2024-03-02T12:09:28.129 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-02T12:09:28.138 DEBUG:teuthology.orchestra.console.smithi165:power on output: Chassis Power Control: Up/On 2024-03-02T12:09:28.139 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-02T12:09:28.145 INFO:teuthology.orchestra.console.smithi173:Power on 2024-03-02T12:09:28.146 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-02T12:09:28.154 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off 2024-03-02T12:09:28.160 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is off 2024-03-02T12:09:28.166 DEBUG:teuthology.orchestra.console.smithi173:power on output: Chassis Power Control: Up/On 2024-03-02T12:09:28.166 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-02T12:09:28.188 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2024-03-02T12:09:32.156 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-02T12:09:32.163 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-02T12:09:32.190 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-02T12:09:32.265 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2024-03-02T12:09:32.272 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is off 2024-03-02T12:09:32.299 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2024-03-02T12:09:32.367 INFO:teuthology.orchestra.console.smithi027:Power on completed 2024-03-02T12:09:32.467 INFO:teuthology.provision.fog.smithi027:Waiting for deploy to finish 2024-03-02T12:09:36.276 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-02T12:09:36.300 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-02T12:09:36.386 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is on 2024-03-02T12:09:36.410 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2024-03-02T12:09:36.487 INFO:teuthology.orchestra.console.smithi165:Power on completed 2024-03-02T12:09:36.511 INFO:teuthology.orchestra.console.smithi173:Power on completed 2024-03-02T12:09:36.588 INFO:teuthology.provision.fog.smithi165:Waiting for deploy to finish 2024-03-02T12:09:36.612 INFO:teuthology.provision.fog.smithi173:Waiting for deploy to finish 2024-03-02T12:12:07.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:12:07.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:12:07.887 ERROR:teuthology.orchestra.connection:Error authenticating with smithi165.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-03-02T12:12:10.876 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-03-02T12:12:16.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:12:19.936 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-03-02T12:12:25.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:12:28.988 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-03-02T12:12:34.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:12:38.048 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-03-02T12:12:44.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:12:46.524 WARNING:teuthology.provision.fog.smithi165:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-03-02T12:12:47.100 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-03-02T12:12:52.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:12:53.101 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:12:55.584 WARNING:teuthology.provision.fog.smithi165:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-03-02T12:12:56.156 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-03-02T12:13:01.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:13:02.157 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:13:03.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:13:04.636 WARNING:teuthology.provision.fog.smithi165:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-03-02T12:13:07.068 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-03-02T12:13:10.637 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:13:13.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:13:16.124 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-03-02T12:13:20.572 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-03-02T12:13:22.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:13:25.180 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-03-02T12:13:26.573 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:13:29.628 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-03-02T12:13:31.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:13:34.236 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-03-02T12:13:35.630 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:13:38.684 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-03-02T12:13:40.237 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:13:43.292 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-03-02T12:13:44.685 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:13:45.018 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-02T12:13:45.018 DEBUG:teuthology.orchestra.run.smithi173:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-02T12:13:49.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:13:52.348 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-03-02T12:13:58.350 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:14:01.408 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-03-02T12:14:05.344 INFO:teuthology.provision.fog.smithi173:Node is ready 2024-03-02T12:14:05.345 DEBUG:teuthology.orchestra.run.smithi173:> hostname 2024-03-02T12:14:05.363 INFO:teuthology.orchestra.run.smithi173.stdout:smithi173.front.sepia.ceph.com 2024-03-02T12:14:05.364 DEBUG:teuthology.orchestra.run.smithi173:> grep smithi173.front.sepia.ceph.com /etc/hosts 2024-03-02T12:14:05.421 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.15.173 smithi173.front.sepia.ceph.com smithi173 2024-03-02T12:14:05.421 DEBUG:teuthology.orchestra.run.smithi173:> sudo hostname smithi173 2024-03-02T12:14:05.547 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/smithi173.front.sepia.ceph.com/smithi173/g' /etc/hosts 2024-03-02T12:14:05.577 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/172.21.15.173/172.21.15.173/g' /etc/hosts 2024-03-02T12:14:05.646 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/smithi173.front.sepia.ceph.com/smithi173/g' /etc/hostname 2024-03-02T12:14:05.714 DEBUG:teuthology.orchestra.run.smithi173:> sudo hostname smithi173 2024-03-02T12:14:05.781 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2024-03-02T12:14:05.836 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2024-03-02T12:14:05.836 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="9" 2024-03-02T12:14:05.836 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2024-03-02T12:14:05.836 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2024-03-02T12:14:05.836 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="9" 2024-03-02T12:14:05.836 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el9" 2024-03-02T12:14:05.837 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-02T12:14:05.837 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2024-03-02T12:14:05.837 INFO:teuthology.orchestra.run.smithi173.stdout:LOGO="fedora-logo-icon" 2024-03-02T12:14:05.837 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-02T12:14:05.837 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2024-03-02T12:14:05.837 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-02T12:14:05.837 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-02T12:14:05.837 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-02T12:14:05.838 INFO:teuthology.provision.fog.smithi173:Deploy complete! 2024-03-02T12:14:05.839 DEBUG:teuthology.parallel:result is None 2024-03-02T12:14:07.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:14:10.460 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-03-02T12:14:10.639 WARNING:teuthology.provision.fog.smithi165:timed out 2024-03-02T12:14:16.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:14:16.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:14:16.852 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-02T12:14:16.852 DEBUG:teuthology.orchestra.run.smithi165:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-02T12:14:17.186 INFO:teuthology.provision.fog.smithi165:Node is ready 2024-03-02T12:14:17.186 DEBUG:teuthology.orchestra.run.smithi165:> hostname 2024-03-02T12:14:17.202 INFO:teuthology.orchestra.run.smithi165.stdout:smithi165.front.sepia.ceph.com 2024-03-02T12:14:17.202 DEBUG:teuthology.orchestra.run.smithi165:> grep smithi165.front.sepia.ceph.com /etc/hosts 2024-03-02T12:14:17.258 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.15.165 smithi165.front.sepia.ceph.com smithi165 2024-03-02T12:14:17.259 DEBUG:teuthology.orchestra.run.smithi165:> sudo hostname smithi165 2024-03-02T12:14:17.361 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i -e 's/smithi165.front.sepia.ceph.com/smithi165/g' /etc/hosts 2024-03-02T12:14:17.390 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i -e 's/172.21.15.165/172.21.15.165/g' /etc/hosts 2024-03-02T12:14:17.459 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i -e 's/smithi165.front.sepia.ceph.com/smithi165/g' /etc/hostname 2024-03-02T12:14:17.527 DEBUG:teuthology.orchestra.run.smithi165:> sudo hostname smithi165 2024-03-02T12:14:17.593 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-03-02T12:14:17.647 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2024-03-02T12:14:17.647 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9" 2024-03-02T12:14:17.648 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2024-03-02T12:14:17.648 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2024-03-02T12:14:17.648 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9" 2024-03-02T12:14:17.648 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9" 2024-03-02T12:14:17.648 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-02T12:14:17.648 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2024-03-02T12:14:17.648 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon" 2024-03-02T12:14:17.648 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-02T12:14:17.648 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2024-03-02T12:14:17.648 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-02T12:14:17.648 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-02T12:14:17.648 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-02T12:14:17.649 INFO:teuthology.provision.fog.smithi165:Deploy complete! 2024-03-02T12:14:17.649 DEBUG:teuthology.parallel:result is None 2024-03-02T12:14:19.516 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-03-02T12:14:25.518 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:15:25.520 WARNING:teuthology.provision.fog.smithi027:timed out 2024-03-02T12:15:31.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:15:31.801 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-02T12:15:31.801 DEBUG:teuthology.orchestra.run.smithi027:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-02T12:15:32.128 INFO:teuthology.provision.fog.smithi027:Node is ready 2024-03-02T12:15:32.129 DEBUG:teuthology.orchestra.run.smithi027:> hostname 2024-03-02T12:15:32.143 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027.front.sepia.ceph.com 2024-03-02T12:15:32.144 DEBUG:teuthology.orchestra.run.smithi027:> grep smithi027.front.sepia.ceph.com /etc/hosts 2024-03-02T12:15:32.199 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27 smithi027.front.sepia.ceph.com smithi027 2024-03-02T12:15:32.199 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname smithi027 2024-03-02T12:15:32.317 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/smithi027.front.sepia.ceph.com/smithi027/g' /etc/hosts 2024-03-02T12:15:32.349 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/172.21.15.27/172.21.15.27/g' /etc/hosts 2024-03-02T12:15:32.420 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/smithi027.front.sepia.ceph.com/smithi027/g' /etc/hostname 2024-03-02T12:15:32.491 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname smithi027 2024-03-02T12:15:32.561 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-03-02T12:15:32.616 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2024-03-02T12:15:32.616 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2024-03-02T12:15:32.616 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2024-03-02T12:15:32.616 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2024-03-02T12:15:32.616 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2024-03-02T12:15:32.616 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2024-03-02T12:15:32.616 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-02T12:15:32.616 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2024-03-02T12:15:32.617 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2024-03-02T12:15:32.617 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-02T12:15:32.617 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2024-03-02T12:15:32.617 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-02T12:15:32.617 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-02T12:15:32.617 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-02T12:15:32.617 INFO:teuthology.provision.fog.smithi027:Deploy complete! 2024-03-02T12:15:32.618 DEBUG:teuthology.parallel:result is None 2024-03-02T12:15:32.832 INFO:teuthology.lock.ops:Checking smithi027.front.sepia.ceph.com 2024-03-02T12:15:32.832 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-02T12:15:32.856 INFO:teuthology.lock.ops:Checking smithi165.front.sepia.ceph.com 2024-03-02T12:15:32.856 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-02T12:15:32.872 INFO:teuthology.lock.ops:Checking smithi173.front.sepia.ceph.com 2024-03-02T12:15:32.873 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-02T12:15:32.890 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: set os type and version on server 2024-03-02T12:15:32.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-03-02T12:15:32.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:15:32.990 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-03-02T12:15:33.058 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-03-02T12:15:33.058 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2024-03-02T12:15:33.113 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2024-03-02T12:15:33.113 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-03-02T12:15:33.168 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2024-03-02T12:15:33.168 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2024-03-02T12:15:33.168 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2024-03-02T12:15:33.168 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2024-03-02T12:15:33.168 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2024-03-02T12:15:33.168 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2024-03-02T12:15:33.168 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-02T12:15:33.168 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2024-03-02T12:15:33.169 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2024-03-02T12:15:33.169 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-02T12:15:33.169 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2024-03-02T12:15:33.169 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-02T12:15:33.169 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-02T12:15:33.169 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-02T12:15:33.169 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-03-02T12:15:33.189 INFO:teuthology.lock.ops:Updating [smithi165.front.sepia.ceph.com]: set os type and version on server 2024-03-02T12:15:33.189 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-03-02T12:15:33.190 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:15:33.276 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-03-02T12:15:33.442 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-03-02T12:15:33.442 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2024-03-02T12:15:33.496 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2024-03-02T12:15:33.496 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-03-02T12:15:33.550 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2024-03-02T12:15:33.550 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9" 2024-03-02T12:15:33.550 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2024-03-02T12:15:33.550 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2024-03-02T12:15:33.550 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9" 2024-03-02T12:15:33.550 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9" 2024-03-02T12:15:33.550 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-02T12:15:33.550 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2024-03-02T12:15:33.550 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon" 2024-03-02T12:15:33.550 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-02T12:15:33.551 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2024-03-02T12:15:33.551 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-02T12:15:33.551 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-02T12:15:33.551 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-02T12:15:33.551 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-03-02T12:15:33.599 INFO:teuthology.lock.ops:Updating [smithi173.front.sepia.ceph.com]: set os type and version on server 2024-03-02T12:15:33.599 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-03-02T12:15:33.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-02T12:15:33.685 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-03-02T12:15:33.854 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-03-02T12:15:33.854 DEBUG:teuthology.orchestra.run.smithi173:> uname -m 2024-03-02T12:15:33.907 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2024-03-02T12:15:33.908 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2024-03-02T12:15:33.962 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2024-03-02T12:15:33.962 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="9" 2024-03-02T12:15:33.962 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2024-03-02T12:15:33.962 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2024-03-02T12:15:33.962 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="9" 2024-03-02T12:15:33.962 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el9" 2024-03-02T12:15:33.962 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-02T12:15:33.962 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2024-03-02T12:15:33.962 INFO:teuthology.orchestra.run.smithi173.stdout:LOGO="fedora-logo-icon" 2024-03-02T12:15:33.962 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-02T12:15:33.963 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2024-03-02T12:15:33.963 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-02T12:15:33.963 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-02T12:15:33.963 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-02T12:15:33.963 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2024-03-02T12:15:33.989 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:15:34.081 INFO:teuthology.dispatcher.supervisor:Running job 7579308 2024-03-02T12:15:34.082 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-03-02_11:23:29-fs-main-2024feb28-testing-default-smithi/7579308 --name rishabh-2024-03-02_11:23:29-fs-main-2024feb28-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/iozone}} -- /home/teuthworker/archive/rishabh-2024-03-02_11:23:29-fs-main-2024feb28-testing-default-smithi/7579308/orig.config.yaml 2024-03-02T12:15:34.085 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2024-03-02_11:23:29-fs-main-2024feb28-testing-default-smithi/7579308 2024-03-02T12:15:34.085 INFO:teuthology.dispatcher.supervisor:Job PID: 3883877 2024-03-02T12:15:34.085 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-03-02T12:17:34.087 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:19:34.159 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:21:34.232 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:23:34.315 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:25:34.386 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:27:34.464 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:29:34.535 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:31:34.608 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:33:34.674 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:35:34.745 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:37:34.829 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:39:34.896 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:41:34.966 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:43:35.044 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:45:35.125 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:47:35.200 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:49:35.273 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:51:35.355 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:53:35.429 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:55:35.491 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:57:35.576 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-02T12:57:35.646 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1