2024-02-18T09:18:55.508 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-18T09:18:55.509 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T09:18:55.689 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8a64d94c10>, signals=[15]) 2024-02-18T09:18:55.690 INFO:teuthology.lock.ops:Start node 'smithi046.front.sepia.ceph.com' reimaging 2024-02-18T09:18:55.690 INFO:teuthology.lock.ops:Updating [smithi046.front.sepia.ceph.com]: reset os type and version on server 2024-02-18T09:18:55.690 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-02-18T09:18:55.711 INFO:teuthology.lock.ops:Start node 'smithi110.front.sepia.ceph.com' reimaging 2024-02-18T09:18:55.711 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: reset os type and version on server 2024-02-18T09:18:55.711 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-02-18T09:18:55.736 INFO:teuthology.lock.ops:Start node 'smithi191.front.sepia.ceph.com' reimaging 2024-02-18T09:18:55.736 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: reset os type and version on server 2024-02-18T09:18:55.736 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-02-18T09:18:55.831 DEBUG:teuthology.provision.fog.smithi046:Requesting image smithi_centos_9.stream (ID 10) 2024-02-18T09:18:55.839 DEBUG:teuthology.provision.fog.smithi110:Requesting image smithi_centos_9.stream (ID 10) 2024-02-18T09:18:55.875 DEBUG:teuthology.provision.fog.smithi191:Requesting image smithi_centos_9.stream (ID 10) 2024-02-18T09:18:55.879 INFO:teuthology.provision.fog.smithi046:Scheduling deploy of centos 9.stream 2024-02-18T09:18:55.883 INFO:teuthology.provision.fog.smithi110:Scheduling deploy of centos 9.stream 2024-02-18T09:18:55.928 INFO:teuthology.provision.fog.smithi191:Scheduling deploy of centos 9.stream 2024-02-18T09:18:56.154 INFO:teuthology.orchestra.console.smithi110:Power off 2024-02-18T09:18:56.154 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-18T09:18:56.167 INFO:teuthology.orchestra.console.smithi046:Power off 2024-02-18T09:18:56.167 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-18T09:18:56.176 DEBUG:teuthology.orchestra.console.smithi110:power off output: Chassis Power Control: Down/Off 2024-02-18T09:18:56.177 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T09:18:56.190 DEBUG:teuthology.orchestra.console.smithi046:power off output: Chassis Power Control: Down/Off 2024-02-18T09:18:56.190 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T09:18:56.198 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off 2024-02-18T09:18:56.216 INFO:teuthology.orchestra.console.smithi191:Power off 2024-02-18T09:18:56.216 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-18T09:18:56.225 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off 2024-02-18T09:18:56.242 DEBUG:teuthology.orchestra.console.smithi191:power off output: Chassis Power Control: Down/Off 2024-02-18T09:18:56.242 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T09:18:56.266 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2024-02-18T09:18:56.299 INFO:teuthology.orchestra.console.smithi110:Power off completed 2024-02-18T09:18:56.326 INFO:teuthology.orchestra.console.smithi046:Power off completed 2024-02-18T09:18:56.368 INFO:teuthology.orchestra.console.smithi191:Power off completed 2024-02-18T09:18:56.400 INFO:teuthology.orchestra.console.smithi110:Power on 2024-02-18T09:18:56.400 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-18T09:18:56.421 DEBUG:teuthology.orchestra.console.smithi110:power on output: Chassis Power Control: Up/On 2024-02-18T09:18:56.422 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T09:18:56.431 INFO:teuthology.orchestra.console.smithi046:Power on 2024-02-18T09:18:56.431 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-18T09:18:56.445 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off 2024-02-18T09:18:56.453 DEBUG:teuthology.orchestra.console.smithi046:power on output: Chassis Power Control: Up/On 2024-02-18T09:18:56.453 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T09:18:56.469 INFO:teuthology.orchestra.console.smithi191:Power on 2024-02-18T09:18:56.469 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-18T09:18:56.478 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off 2024-02-18T09:18:56.490 DEBUG:teuthology.orchestra.console.smithi191:power on output: Chassis Power Control: Up/On 2024-02-18T09:18:56.490 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T09:18:56.510 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2024-02-18T09:19:00.451 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T09:19:00.479 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T09:19:00.511 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T09:19:00.560 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on 2024-02-18T09:19:00.587 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2024-02-18T09:19:00.622 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2024-02-18T09:19:00.661 INFO:teuthology.orchestra.console.smithi110:Power on completed 2024-02-18T09:19:00.688 INFO:teuthology.orchestra.console.smithi046:Power on completed 2024-02-18T09:19:00.762 INFO:teuthology.provision.fog.smithi110:Waiting for deploy to finish 2024-02-18T09:19:00.789 INFO:teuthology.provision.fog.smithi046:Waiting for deploy to finish 2024-02-18T09:19:04.626 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T09:19:04.736 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-02-18T09:19:04.837 INFO:teuthology.orchestra.console.smithi191:Power on completed 2024-02-18T09:19:04.938 INFO:teuthology.provision.fog.smithi191:Waiting for deploy to finish 2024-02-18T09:22:02.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:22:05.244 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-02-18T09:22:06.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:22:09.312 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-02-18T09:22:11.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:22:14.303 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-02-18T09:22:15.314 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:22:17.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:22:18.396 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-02-18T09:22:20.284 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-02-18T09:22:20.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:22:23.356 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-02-18T09:22:24.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:22:26.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:22:27.452 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-02-18T09:22:29.340 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-02-18T09:22:29.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:22:32.412 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-02-18T09:22:33.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:22:35.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:22:36.508 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-02-18T09:22:38.396 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-02-18T09:22:38.414 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:22:41.475 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-02-18T09:22:42.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:22:44.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:22:45.564 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-02-18T09:22:47.456 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-02-18T09:22:47.476 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:22:50.556 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-02-18T09:22:51.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:22:53.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:22:54.620 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-02-18T09:22:56.540 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-02-18T09:22:56.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:23:00.625 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:23:02.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:23:05.596 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-02-18T09:23:11.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:23:56.600 WARNING:teuthology.provision.fog.smithi110:timed out 2024-02-18T09:24:00.632 WARNING:teuthology.provision.fog.smithi191:timed out 2024-02-18T09:24:02.604 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:24:02.826 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-18T09:24:02.827 DEBUG:teuthology.orchestra.run.smithi110:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-18T09:24:03.158 INFO:teuthology.provision.fog.smithi110:Node is ready 2024-02-18T09:24:03.158 DEBUG:teuthology.orchestra.run.smithi110:> hostname 2024-02-18T09:24:03.173 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110.front.sepia.ceph.com 2024-02-18T09:24:03.173 DEBUG:teuthology.orchestra.run.smithi110:> grep smithi110.front.sepia.ceph.com /etc/hosts 2024-02-18T09:24:03.227 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110 smithi110.front.sepia.ceph.com smithi110 2024-02-18T09:24:03.228 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname smithi110 2024-02-18T09:24:03.354 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i -e 's/smithi110.front.sepia.ceph.com/smithi110/g' /etc/hosts 2024-02-18T09:24:03.381 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i -e 's/172.21.15.110/172.21.15.110/g' /etc/hosts 2024-02-18T09:24:03.449 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i -e 's/smithi110.front.sepia.ceph.com/smithi110/g' /etc/hostname 2024-02-18T09:24:03.516 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname smithi110 2024-02-18T09:24:03.582 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-02-18T09:24:03.637 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream" 2024-02-18T09:24:03.637 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9" 2024-02-18T09:24:03.637 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2024-02-18T09:24:03.637 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2024-02-18T09:24:03.637 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9" 2024-02-18T09:24:03.637 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9" 2024-02-18T09:24:03.637 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-18T09:24:03.637 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2024-02-18T09:24:03.637 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon" 2024-02-18T09:24:03.637 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-18T09:24:03.637 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2024-02-18T09:24:03.638 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-18T09:24:03.638 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-18T09:24:03.638 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-18T09:24:03.638 INFO:teuthology.provision.fog.smithi110:Deploy complete! 2024-02-18T09:24:03.640 DEBUG:teuthology.parallel:result is None 2024-02-18T09:24:06.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:24:07.056 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-18T09:24:07.057 DEBUG:teuthology.orchestra.run.smithi191:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-18T09:24:07.380 INFO:teuthology.provision.fog.smithi191:Node is ready 2024-02-18T09:24:07.381 DEBUG:teuthology.orchestra.run.smithi191:> hostname 2024-02-18T09:24:07.395 INFO:teuthology.orchestra.run.smithi191.stdout:smithi191.front.sepia.ceph.com 2024-02-18T09:24:07.395 DEBUG:teuthology.orchestra.run.smithi191:> grep smithi191.front.sepia.ceph.com /etc/hosts 2024-02-18T09:24:07.450 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.15.191 smithi191.front.sepia.ceph.com smithi191 2024-02-18T09:24:07.450 DEBUG:teuthology.orchestra.run.smithi191:> sudo hostname smithi191 2024-02-18T09:24:07.575 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i -e 's/smithi191.front.sepia.ceph.com/smithi191/g' /etc/hosts 2024-02-18T09:24:07.603 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i -e 's/172.21.15.191/172.21.15.191/g' /etc/hosts 2024-02-18T09:24:07.670 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i -e 's/smithi191.front.sepia.ceph.com/smithi191/g' /etc/hostname 2024-02-18T09:24:07.737 DEBUG:teuthology.orchestra.run.smithi191:> sudo hostname smithi191 2024-02-18T09:24:07.803 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-02-18T09:24:07.858 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream" 2024-02-18T09:24:07.858 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="9" 2024-02-18T09:24:07.858 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos" 2024-02-18T09:24:07.858 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora" 2024-02-18T09:24:07.858 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="9" 2024-02-18T09:24:07.858 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el9" 2024-02-18T09:24:07.858 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-18T09:24:07.858 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2024-02-18T09:24:07.858 INFO:teuthology.orchestra.run.smithi191.stdout:LOGO="fedora-logo-icon" 2024-02-18T09:24:07.858 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-18T09:24:07.858 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/" 2024-02-18T09:24:07.858 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-18T09:24:07.858 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-18T09:24:07.859 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-18T09:24:07.859 INFO:teuthology.provision.fog.smithi191:Deploy complete! 2024-02-18T09:24:07.859 DEBUG:teuthology.parallel:result is None 2024-02-18T09:24:11.609 WARNING:teuthology.provision.fog.smithi046:timed out 2024-02-18T09:24:17.616 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:24:17.865 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-18T09:24:17.865 DEBUG:teuthology.orchestra.run.smithi046:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-18T09:24:18.225 INFO:teuthology.provision.fog.smithi046:Node is ready 2024-02-18T09:24:18.225 DEBUG:teuthology.orchestra.run.smithi046:> hostname 2024-02-18T09:24:18.240 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046.front.sepia.ceph.com 2024-02-18T09:24:18.241 DEBUG:teuthology.orchestra.run.smithi046:> grep smithi046.front.sepia.ceph.com /etc/hosts 2024-02-18T09:24:18.297 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2024-02-18T09:24:18.297 DEBUG:teuthology.orchestra.run.smithi046:> sudo hostname smithi046 2024-02-18T09:24:18.420 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i -e 's/smithi046.front.sepia.ceph.com/smithi046/g' /etc/hosts 2024-02-18T09:24:18.453 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i -e 's/172.21.15.46/172.21.15.46/g' /etc/hosts 2024-02-18T09:24:18.525 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i -e 's/smithi046.front.sepia.ceph.com/smithi046/g' /etc/hostname 2024-02-18T09:24:18.596 DEBUG:teuthology.orchestra.run.smithi046:> sudo hostname smithi046 2024-02-18T09:24:18.667 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-02-18T09:24:18.722 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2024-02-18T09:24:18.722 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="9" 2024-02-18T09:24:18.722 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2024-02-18T09:24:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2024-02-18T09:24:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="9" 2024-02-18T09:24:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el9" 2024-02-18T09:24:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-18T09:24:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2024-02-18T09:24:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:LOGO="fedora-logo-icon" 2024-02-18T09:24:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-18T09:24:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2024-02-18T09:24:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-18T09:24:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-18T09:24:18.723 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-18T09:24:18.723 INFO:teuthology.provision.fog.smithi046:Deploy complete! 2024-02-18T09:24:18.724 DEBUG:teuthology.parallel:result is None 2024-02-18T09:24:18.939 INFO:teuthology.lock.ops:Checking smithi046.front.sepia.ceph.com 2024-02-18T09:24:18.940 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-18T09:24:18.964 INFO:teuthology.lock.ops:Checking smithi110.front.sepia.ceph.com 2024-02-18T09:24:18.964 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-18T09:24:18.980 INFO:teuthology.lock.ops:Checking smithi191.front.sepia.ceph.com 2024-02-18T09:24:18.980 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-18T09:24:18.998 INFO:teuthology.lock.ops:Updating [smithi046.front.sepia.ceph.com]: set os type and version on server 2024-02-18T09:24:18.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-02-18T09:24:18.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:24:19.096 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-02-18T09:24:19.161 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-02-18T09:24:19.162 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-02-18T09:24:19.216 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-02-18T09:24:19.216 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-02-18T09:24:19.270 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2024-02-18T09:24:19.270 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="9" 2024-02-18T09:24:19.270 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2024-02-18T09:24:19.270 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2024-02-18T09:24:19.270 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="9" 2024-02-18T09:24:19.270 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el9" 2024-02-18T09:24:19.270 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-18T09:24:19.271 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2024-02-18T09:24:19.271 INFO:teuthology.orchestra.run.smithi046.stdout:LOGO="fedora-logo-icon" 2024-02-18T09:24:19.271 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-18T09:24:19.271 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2024-02-18T09:24:19.271 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-18T09:24:19.271 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-18T09:24:19.271 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-18T09:24:19.271 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-02-18T09:24:19.289 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: set os type and version on server 2024-02-18T09:24:19.289 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-02-18T09:24:19.289 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:24:19.377 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-02-18T09:24:19.546 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-02-18T09:24:19.546 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2024-02-18T09:24:19.600 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2024-02-18T09:24:19.600 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-02-18T09:24:19.654 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream" 2024-02-18T09:24:19.654 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9" 2024-02-18T09:24:19.654 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2024-02-18T09:24:19.654 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2024-02-18T09:24:19.654 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9" 2024-02-18T09:24:19.654 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9" 2024-02-18T09:24:19.654 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-18T09:24:19.654 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2024-02-18T09:24:19.654 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon" 2024-02-18T09:24:19.654 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-18T09:24:19.654 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2024-02-18T09:24:19.655 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-18T09:24:19.655 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-18T09:24:19.655 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-18T09:24:19.655 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-02-18T09:24:19.672 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: set os type and version on server 2024-02-18T09:24:19.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-18T09:24:19.672 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T09:24:19.758 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-02-18T09:24:19.882 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-18T09:24:19.882 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2024-02-18T09:24:19.936 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2024-02-18T09:24:19.936 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-02-18T09:24:19.990 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream" 2024-02-18T09:24:19.990 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="9" 2024-02-18T09:24:19.990 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos" 2024-02-18T09:24:19.990 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora" 2024-02-18T09:24:19.990 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="9" 2024-02-18T09:24:19.990 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el9" 2024-02-18T09:24:19.990 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-18T09:24:19.990 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2024-02-18T09:24:19.991 INFO:teuthology.orchestra.run.smithi191.stdout:LOGO="fedora-logo-icon" 2024-02-18T09:24:19.991 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-18T09:24:19.991 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/" 2024-02-18T09:24:19.991 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-18T09:24:19.991 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-18T09:24:19.991 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-18T09:24:19.991 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-02-18T09:24:20.014 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T09:24:20.110 INFO:teuthology.dispatcher.supervisor:Running job 7564219 2024-02-18T09:24:20.110 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564219 --name vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/blogbench}} -- /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564219/orig.config.yaml 2024-02-18T09:24:20.112 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564219 2024-02-18T09:24:20.113 INFO:teuthology.dispatcher.supervisor:Job PID: 1646456 2024-02-18T09:24:20.113 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-02-18T09:26:20.189 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T09:28:20.408 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T09:30:20.577 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T09:32:20.740 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T09:34:20.987 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T09:36:21.155 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T09:38:21.320 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T09:38:21.392 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1