2024-04-17T10:17:33.781 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-17T10:17:33.781 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T10:17:33.953 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7dad958c10>, signals=[15]) 2024-04-17T10:17:33.953 INFO:teuthology.lock.ops:Start node 'smithi047.front.sepia.ceph.com' reimaging 2024-04-17T10:17:33.953 INFO:teuthology.lock.ops:Updating [smithi047.front.sepia.ceph.com]: reset os type and version on server 2024-04-17T10:17:33.953 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-04-17T10:17:33.977 INFO:teuthology.lock.ops:Start node 'smithi055.front.sepia.ceph.com' reimaging 2024-04-17T10:17:33.977 INFO:teuthology.lock.ops:Updating [smithi055.front.sepia.ceph.com]: reset os type and version on server 2024-04-17T10:17:33.977 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2024-04-17T10:17:34.001 INFO:teuthology.lock.ops:Start node 'smithi138.front.sepia.ceph.com' reimaging 2024-04-17T10:17:34.002 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: reset os type and version on server 2024-04-17T10:17:34.002 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-04-17T10:17:34.102 DEBUG:teuthology.provision.fog.smithi047:Requesting image smithi_centos_9.stream (ID 10) 2024-04-17T10:17:34.109 DEBUG:teuthology.provision.fog.smithi055:Requesting image smithi_centos_9.stream (ID 10) 2024-04-17T10:17:34.117 DEBUG:teuthology.provision.fog.smithi138:Requesting image smithi_centos_9.stream (ID 10) 2024-04-17T10:17:34.146 INFO:teuthology.provision.fog.smithi055:Scheduling deploy of centos 9.stream 2024-04-17T10:17:34.150 INFO:teuthology.provision.fog.smithi047:Scheduling deploy of centos 9.stream 2024-04-17T10:17:34.169 INFO:teuthology.provision.fog.smithi138:Scheduling deploy of centos 9.stream 2024-04-17T10:17:34.481 INFO:teuthology.orchestra.console.smithi138:Power off 2024-04-17T10:17:34.481 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T10:17:34.495 INFO:teuthology.orchestra.console.smithi055:Power off 2024-04-17T10:17:34.495 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T10:17:34.506 INFO:teuthology.orchestra.console.smithi047:Power off 2024-04-17T10:17:34.506 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T10:17:34.514 DEBUG:teuthology.orchestra.console.smithi138:power off output: Chassis Power Control: Down/Off 2024-04-17T10:17:34.515 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:17:34.523 DEBUG:teuthology.orchestra.console.smithi055:power off output: Chassis Power Control: Down/Off 2024-04-17T10:17:34.523 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:17:34.531 DEBUG:teuthology.orchestra.console.smithi047:power off output: Chassis Power Control: Down/Off 2024-04-17T10:17:34.531 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:17:34.538 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-04-17T10:17:34.548 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off 2024-04-17T10:17:34.552 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is on 2024-04-17T10:17:34.649 INFO:teuthology.orchestra.console.smithi055:Power off completed 2024-04-17T10:17:34.749 INFO:teuthology.orchestra.console.smithi055:Power on 2024-04-17T10:17:34.750 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-17T10:17:34.771 DEBUG:teuthology.orchestra.console.smithi055:power on output: Chassis Power Control: Up/On 2024-04-17T10:17:34.771 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:17:34.791 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off 2024-04-17T10:17:38.542 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:17:38.554 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:17:38.651 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2024-04-17T10:17:38.662 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is off 2024-04-17T10:17:38.752 INFO:teuthology.orchestra.console.smithi138:Power off completed 2024-04-17T10:17:38.762 INFO:teuthology.orchestra.console.smithi047:Power off completed 2024-04-17T10:17:38.792 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:17:38.853 INFO:teuthology.orchestra.console.smithi138:Power on 2024-04-17T10:17:38.854 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-17T10:17:38.863 INFO:teuthology.orchestra.console.smithi047:Power on 2024-04-17T10:17:38.863 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-17T10:17:38.874 DEBUG:teuthology.orchestra.console.smithi138:power on output: Chassis Power Control: Up/On 2024-04-17T10:17:38.874 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:17:38.884 DEBUG:teuthology.orchestra.console.smithi047:power on output: Chassis Power Control: Up/On 2024-04-17T10:17:38.884 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:17:38.894 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2024-04-17T10:17:38.899 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off 2024-04-17T10:17:38.903 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is off 2024-04-17T10:17:42.897 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:17:42.905 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:17:42.910 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:17:43.007 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2024-04-17T10:17:43.013 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2024-04-17T10:17:43.018 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is off 2024-04-17T10:17:43.114 INFO:teuthology.orchestra.console.smithi055:Power on completed 2024-04-17T10:17:43.214 INFO:teuthology.provision.fog.smithi055:Waiting for deploy to finish 2024-04-17T10:17:47.011 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:17:47.018 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T10:17:47.121 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-04-17T10:17:47.126 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is on 2024-04-17T10:17:47.222 INFO:teuthology.orchestra.console.smithi138:Power on completed 2024-04-17T10:17:47.227 INFO:teuthology.orchestra.console.smithi047:Power on completed 2024-04-17T10:17:47.323 INFO:teuthology.provision.fog.smithi138:Waiting for deploy to finish 2024-04-17T10:17:47.327 INFO:teuthology.provision.fog.smithi047:Waiting for deploy to finish 2024-04-17T10:19:59.549 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:19:59.674 ERROR:teuthology.orchestra.connection:Error authenticating with smithi055.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-04-17T10:21:03.678 WARNING:teuthology.provision.fog.smithi055:timed out 2024-04-17T10:21:04.012 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:21:07.070 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-04-17T10:21:09.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:21:12.734 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55 2024-04-17T10:21:13.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:21:16.126 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-04-17T10:21:18.736 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:21:19.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:21:21.790 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55 2024-04-17T10:21:22.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:21:22.174 WARNING:teuthology.provision.fog.smithi047:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-04-17T10:21:25.182 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-04-17T10:21:27.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:21:28.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:21:28.840 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-17T10:21:28.840 DEBUG:teuthology.orchestra.run.smithi055:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-17T10:21:31.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:21:31.234 WARNING:teuthology.provision.fog.smithi047:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-04-17T10:21:34.238 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-04-17T10:21:37.236 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:21:40.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:21:40.286 WARNING:teuthology.provision.fog.smithi047:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-04-17T10:21:43.294 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-04-17T10:21:46.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:21:49.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:21:49.346 WARNING:teuthology.provision.fog.smithi047:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-04-17T10:21:49.778 INFO:teuthology.provision.fog.smithi055:Node is ready 2024-04-17T10:21:49.778 DEBUG:teuthology.orchestra.run.smithi055:> hostname 2024-04-17T10:21:49.797 INFO:teuthology.orchestra.run.smithi055.stdout:smithi055.front.sepia.ceph.com 2024-04-17T10:21:49.797 DEBUG:teuthology.orchestra.run.smithi055:> grep smithi055.front.sepia.ceph.com /etc/hosts 2024-04-17T10:21:49.858 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55 smithi055.front.sepia.ceph.com smithi055 2024-04-17T10:21:49.858 DEBUG:teuthology.orchestra.run.smithi055:> sudo hostname smithi055 2024-04-17T10:21:50.005 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i -e 's/smithi055.front.sepia.ceph.com/smithi055/g' /etc/hosts 2024-04-17T10:21:50.041 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i -e 's/172.21.15.55/172.21.15.55/g' /etc/hosts 2024-04-17T10:21:50.114 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i -e 's/smithi055.front.sepia.ceph.com/smithi055/g' /etc/hostname 2024-04-17T10:21:50.188 DEBUG:teuthology.orchestra.run.smithi055:> sudo hostname smithi055 2024-04-17T10:21:50.260 DEBUG:teuthology.orchestra.run.smithi055:> cat /etc/os-release 2024-04-17T10:21:50.316 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="CentOS Stream" 2024-04-17T10:21:50.316 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="9" 2024-04-17T10:21:50.316 INFO:teuthology.orchestra.run.smithi055.stdout:ID="centos" 2024-04-17T10:21:50.316 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="rhel fedora" 2024-04-17T10:21:50.316 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="9" 2024-04-17T10:21:50.316 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el9" 2024-04-17T10:21:50.316 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T10:21:50.316 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31" 2024-04-17T10:21:50.316 INFO:teuthology.orchestra.run.smithi055.stdout:LOGO="fedora-logo-icon" 2024-04-17T10:21:50.316 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T10:21:50.317 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://centos.org/" 2024-04-17T10:21:50.317 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T10:21:50.317 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T10:21:50.317 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T10:21:50.317 INFO:teuthology.provision.fog.smithi055:Deploy complete! 2024-04-17T10:21:50.318 DEBUG:teuthology.parallel:result is None 2024-04-17T10:21:52.350 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-04-17T10:21:55.347 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:21:58.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:21:58.398 WARNING:teuthology.provision.fog.smithi047:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-04-17T10:22:01.406 WARNING:teuthology.provision.fog.smithi138:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-04-17T10:22:04.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:22:07.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:23:04.402 WARNING:teuthology.provision.fog.smithi047:timed out 2024-04-17T10:23:07.410 WARNING:teuthology.provision.fog.smithi138:timed out 2024-04-17T10:23:10.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:23:10.678 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-17T10:23:10.678 DEBUG:teuthology.orchestra.run.smithi047:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-17T10:23:11.032 INFO:teuthology.provision.fog.smithi047:Node is ready 2024-04-17T10:23:11.032 DEBUG:teuthology.orchestra.run.smithi047:> hostname 2024-04-17T10:23:11.049 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047.front.sepia.ceph.com 2024-04-17T10:23:11.049 DEBUG:teuthology.orchestra.run.smithi047:> grep smithi047.front.sepia.ceph.com /etc/hosts 2024-04-17T10:23:11.105 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47 smithi047.front.sepia.ceph.com smithi047 2024-04-17T10:23:11.106 DEBUG:teuthology.orchestra.run.smithi047:> sudo hostname smithi047 2024-04-17T10:23:11.207 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i -e 's/smithi047.front.sepia.ceph.com/smithi047/g' /etc/hosts 2024-04-17T10:23:11.240 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i -e 's/172.21.15.47/172.21.15.47/g' /etc/hosts 2024-04-17T10:23:11.314 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i -e 's/smithi047.front.sepia.ceph.com/smithi047/g' /etc/hostname 2024-04-17T10:23:11.386 DEBUG:teuthology.orchestra.run.smithi047:> sudo hostname smithi047 2024-04-17T10:23:11.458 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-04-17T10:23:11.514 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Stream" 2024-04-17T10:23:11.514 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="9" 2024-04-17T10:23:11.514 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2024-04-17T10:23:11.514 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2024-04-17T10:23:11.515 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="9" 2024-04-17T10:23:11.515 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el9" 2024-04-17T10:23:11.515 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T10:23:11.515 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-04-17T10:23:11.515 INFO:teuthology.orchestra.run.smithi047.stdout:LOGO="fedora-logo-icon" 2024-04-17T10:23:11.515 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T10:23:11.515 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://centos.org/" 2024-04-17T10:23:11.515 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T10:23:11.515 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T10:23:11.515 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T10:23:11.515 INFO:teuthology.provision.fog.smithi047:Deploy complete! 2024-04-17T10:23:11.516 DEBUG:teuthology.parallel:result is None 2024-04-17T10:23:13.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:23:13.661 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-17T10:23:13.661 DEBUG:teuthology.orchestra.run.smithi138:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-17T10:23:13.996 INFO:teuthology.provision.fog.smithi138:Node is ready 2024-04-17T10:23:13.996 DEBUG:teuthology.orchestra.run.smithi138:> hostname 2024-04-17T10:23:14.010 INFO:teuthology.orchestra.run.smithi138.stdout:smithi138.front.sepia.ceph.com 2024-04-17T10:23:14.011 DEBUG:teuthology.orchestra.run.smithi138:> grep smithi138.front.sepia.ceph.com /etc/hosts 2024-04-17T10:23:14.065 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138 smithi138.front.sepia.ceph.com smithi138 2024-04-17T10:23:14.066 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname smithi138 2024-04-17T10:23:14.164 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i -e 's/smithi138.front.sepia.ceph.com/smithi138/g' /etc/hosts 2024-04-17T10:23:14.192 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i -e 's/172.21.15.138/172.21.15.138/g' /etc/hosts 2024-04-17T10:23:14.259 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i -e 's/smithi138.front.sepia.ceph.com/smithi138/g' /etc/hostname 2024-04-17T10:23:14.327 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname smithi138 2024-04-17T10:23:14.393 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-04-17T10:23:14.447 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-04-17T10:23:14.447 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-04-17T10:23:14.447 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-04-17T10:23:14.447 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-04-17T10:23:14.447 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-04-17T10:23:14.447 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-04-17T10:23:14.447 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T10:23:14.447 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-04-17T10:23:14.448 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-04-17T10:23:14.448 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T10:23:14.448 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-04-17T10:23:14.448 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T10:23:14.448 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T10:23:14.448 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T10:23:14.448 INFO:teuthology.provision.fog.smithi138:Deploy complete! 2024-04-17T10:23:14.448 DEBUG:teuthology.parallel:result is None 2024-04-17T10:23:14.699 INFO:teuthology.lock.ops:Checking smithi047.front.sepia.ceph.com 2024-04-17T10:23:14.699 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-17T10:23:14.720 INFO:teuthology.lock.ops:Checking smithi055.front.sepia.ceph.com 2024-04-17T10:23:14.720 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-17T10:23:14.736 INFO:teuthology.lock.ops:Checking smithi138.front.sepia.ceph.com 2024-04-17T10:23:14.736 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-17T10:23:14.763 INFO:teuthology.lock.ops:Updating [smithi047.front.sepia.ceph.com]: set os type and version on server 2024-04-17T10:23:14.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-17T10:23:14.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:23:14.868 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-04-17T10:23:14.939 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-17T10:23:14.939 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-04-17T10:23:14.998 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-04-17T10:23:14.999 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-04-17T10:23:15.058 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Stream" 2024-04-17T10:23:15.058 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="9" 2024-04-17T10:23:15.058 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2024-04-17T10:23:15.058 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2024-04-17T10:23:15.058 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="9" 2024-04-17T10:23:15.059 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el9" 2024-04-17T10:23:15.059 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T10:23:15.059 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-04-17T10:23:15.059 INFO:teuthology.orchestra.run.smithi047.stdout:LOGO="fedora-logo-icon" 2024-04-17T10:23:15.059 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T10:23:15.059 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://centos.org/" 2024-04-17T10:23:15.059 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T10:23:15.059 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T10:23:15.059 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T10:23:15.060 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-04-17T10:23:15.079 INFO:teuthology.lock.ops:Updating [smithi055.front.sepia.ceph.com]: set os type and version on server 2024-04-17T10:23:15.079 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-04-17T10:23:15.079 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:23:15.180 DEBUG:teuthology.orchestra.run.smithi055:> true 2024-04-17T10:23:15.360 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-04-17T10:23:15.360 DEBUG:teuthology.orchestra.run.smithi055:> uname -m 2024-04-17T10:23:15.415 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64 2024-04-17T10:23:15.415 DEBUG:teuthology.orchestra.run.smithi055:> cat /etc/os-release 2024-04-17T10:23:15.469 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="CentOS Stream" 2024-04-17T10:23:15.470 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="9" 2024-04-17T10:23:15.470 INFO:teuthology.orchestra.run.smithi055.stdout:ID="centos" 2024-04-17T10:23:15.470 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="rhel fedora" 2024-04-17T10:23:15.470 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="9" 2024-04-17T10:23:15.470 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el9" 2024-04-17T10:23:15.470 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T10:23:15.470 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31" 2024-04-17T10:23:15.470 INFO:teuthology.orchestra.run.smithi055.stdout:LOGO="fedora-logo-icon" 2024-04-17T10:23:15.470 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T10:23:15.470 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://centos.org/" 2024-04-17T10:23:15.470 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T10:23:15.470 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T10:23:15.470 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T10:23:15.471 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2024-04-17T10:23:15.496 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: set os type and version on server 2024-04-17T10:23:15.496 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-17T10:23:15.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T10:23:15.588 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-04-17T10:23:15.663 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-17T10:23:15.663 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-04-17T10:23:15.719 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-04-17T10:23:15.719 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-04-17T10:23:15.776 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-04-17T10:23:15.776 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-04-17T10:23:15.776 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-04-17T10:23:15.776 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-04-17T10:23:15.776 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-04-17T10:23:15.776 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-04-17T10:23:15.776 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T10:23:15.776 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-04-17T10:23:15.776 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-04-17T10:23:15.777 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T10:23:15.777 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-04-17T10:23:15.777 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T10:23:15.777 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T10:23:15.777 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T10:23:15.777 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-04-17T10:23:15.796 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T10:23:15.899 INFO:teuthology.dispatcher.supervisor:Running job 7660062 2024-04-17T10:23:15.899 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660062 --name rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/norstats}} -- /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660062/orig.config.yaml 2024-04-17T10:23:15.902 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660062 2024-04-17T10:23:15.902 INFO:teuthology.dispatcher.supervisor:Job PID: 841859 2024-04-17T10:23:15.902 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-17T10:25:15.904 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T10:27:15.968 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T10:29:16.034 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T10:31:16.099 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T10:33:16.171 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T10:33:16.243 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2024-04-17T10:33:16.423 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi047 but it was locked by another job: /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660077 2024-04-17T10:33:16.424 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi055 but it was locked by another job: /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660076 2024-04-17T10:33:16.424 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi138 but it was locked by another job: /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7660076