2024-03-21T17:22:38.407 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-21T17:22:38.407 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T17:22:38.597 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff540c3ac10>, signals=[15]) 2024-03-21T17:22:38.597 INFO:teuthology.lock.ops:Start node 'smithi084.front.sepia.ceph.com' reimaging 2024-03-21T17:22:38.597 INFO:teuthology.lock.ops:Updating [smithi084.front.sepia.ceph.com]: reset os type and version on server 2024-03-21T17:22:38.597 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2024-03-21T17:22:38.614 INFO:teuthology.lock.ops:Start node 'smithi088.front.sepia.ceph.com' reimaging 2024-03-21T17:22:38.614 INFO:teuthology.lock.ops:Updating [smithi088.front.sepia.ceph.com]: reset os type and version on server 2024-03-21T17:22:38.615 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2024-03-21T17:22:38.639 INFO:teuthology.lock.ops:Start node 'smithi112.front.sepia.ceph.com' reimaging 2024-03-21T17:22:38.640 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: reset os type and version on server 2024-03-21T17:22:38.640 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-03-21T17:22:38.730 DEBUG:teuthology.provision.fog.smithi084:Requesting image smithi_centos_9.stream (ID 10) 2024-03-21T17:22:38.739 DEBUG:teuthology.provision.fog.smithi088:Requesting image smithi_centos_9.stream (ID 10) 2024-03-21T17:22:38.751 DEBUG:teuthology.provision.fog.smithi112:Requesting image smithi_centos_9.stream (ID 10) 2024-03-21T17:22:38.777 INFO:teuthology.provision.fog.smithi084:Scheduling deploy of centos 9.stream 2024-03-21T17:22:38.780 INFO:teuthology.provision.fog.smithi088:Scheduling deploy of centos 9.stream 2024-03-21T17:22:38.786 INFO:teuthology.provision.fog.smithi112:Scheduling deploy of centos 9.stream 2024-03-21T17:22:39.075 INFO:teuthology.orchestra.console.smithi112:Power off 2024-03-21T17:22:39.076 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T17:22:39.090 INFO:teuthology.orchestra.console.smithi084:Power off 2024-03-21T17:22:39.090 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T17:22:39.104 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off 2024-03-21T17:22:39.104 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T17:22:39.112 DEBUG:teuthology.orchestra.console.smithi084:power off output: Chassis Power Control: Down/Off 2024-03-21T17:22:39.113 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T17:22:39.124 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-03-21T17:22:39.135 INFO:teuthology.orchestra.console.smithi088:Power off 2024-03-21T17:22:39.135 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T17:22:39.145 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is off 2024-03-21T17:22:39.159 DEBUG:teuthology.orchestra.console.smithi088:power off output: Chassis Power Control: Down/Off 2024-03-21T17:22:39.159 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T17:22:39.181 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off 2024-03-21T17:22:39.225 INFO:teuthology.orchestra.console.smithi112:Power off completed 2024-03-21T17:22:39.246 INFO:teuthology.orchestra.console.smithi084:Power off completed 2024-03-21T17:22:39.282 INFO:teuthology.orchestra.console.smithi088:Power off completed 2024-03-21T17:22:39.325 INFO:teuthology.orchestra.console.smithi112:Power on 2024-03-21T17:22:39.326 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-21T17:22:39.348 INFO:teuthology.orchestra.console.smithi084:Power on 2024-03-21T17:22:39.348 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-21T17:22:39.357 DEBUG:teuthology.orchestra.console.smithi112:power on output: Chassis Power Control: Up/On 2024-03-21T17:22:39.357 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T17:22:39.370 DEBUG:teuthology.orchestra.console.smithi084:power on output: Chassis Power Control: Up/On 2024-03-21T17:22:39.370 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T17:22:39.381 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-03-21T17:22:39.383 INFO:teuthology.orchestra.console.smithi088:Power on 2024-03-21T17:22:39.383 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-21T17:22:39.394 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is off 2024-03-21T17:22:39.404 DEBUG:teuthology.orchestra.console.smithi088:power on output: Chassis Power Control: Up/On 2024-03-21T17:22:39.404 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T17:22:39.428 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off 2024-03-21T17:22:43.386 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T17:22:43.395 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T17:22:43.429 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T17:22:43.495 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-03-21T17:22:43.504 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is on 2024-03-21T17:22:43.535 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off 2024-03-21T17:22:43.605 INFO:teuthology.orchestra.console.smithi084:Power on completed 2024-03-21T17:22:43.705 INFO:teuthology.provision.fog.smithi084:Waiting for deploy to finish 2024-03-21T17:22:47.499 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T17:22:47.537 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T17:22:47.625 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-03-21T17:22:47.645 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-03-21T17:22:47.726 INFO:teuthology.orchestra.console.smithi112:Power on completed 2024-03-21T17:22:47.745 INFO:teuthology.orchestra.console.smithi088:Power on completed 2024-03-21T17:22:47.827 INFO:teuthology.provision.fog.smithi112:Waiting for deploy to finish 2024-03-21T17:22:47.846 INFO:teuthology.provision.fog.smithi088:Waiting for deploy to finish 2024-03-21T17:25:19.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:25:22.174 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-03-21T17:25:28.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:25:31.230 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-03-21T17:25:37.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:25:40.290 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-03-21T17:25:46.292 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:25:49.378 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-03-21T17:25:55.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:25:58.462 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-03-21T17:26:04.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:26:04.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:26:07.422 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-03-21T17:26:07.518 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-03-21T17:26:13.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:26:13.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:26:15.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:26:16.510 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-03-21T17:26:18.434 WARNING:teuthology.provision.fog.smithi084:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-03-21T17:26:22.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:26:24.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:26:25.570 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-03-21T17:26:27.522 WARNING:teuthology.provision.fog.smithi084:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-03-21T17:26:31.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:26:31.966 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-03-21T17:26:33.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:26:34.658 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-03-21T17:26:36.610 WARNING:teuthology.provision.fog.smithi084:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-03-21T17:26:37.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:26:40.662 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:26:41.026 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-03-21T17:26:42.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:26:43.742 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-03-21T17:26:45.694 WARNING:teuthology.provision.fog.smithi084:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-03-21T17:26:47.029 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:26:49.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:26:50.110 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-03-21T17:26:51.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:26:52.798 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-03-21T17:26:54.750 WARNING:teuthology.provision.fog.smithi084:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-03-21T17:26:56.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:26:56.603 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-21T17:26:56.604 DEBUG:teuthology.orchestra.run.smithi112:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-21T17:26:58.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:27:00.751 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:27:01.854 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-03-21T17:27:03.810 WARNING:teuthology.provision.fog.smithi084:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-03-21T17:27:07.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:27:09.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:27:10.910 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-03-21T17:27:11.915 INFO:teuthology.provision.fog.smithi112:Node is ready 2024-03-21T17:27:11.915 DEBUG:teuthology.orchestra.run.smithi112:> hostname 2024-03-21T17:27:11.933 INFO:teuthology.orchestra.run.smithi112.stdout:smithi046 2024-03-21T17:27:11.933 DEBUG:teuthology.orchestra.run.smithi112:> grep smithi046 /etc/hosts 2024-03-21T17:27:11.990 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2024-03-21T17:27:11.990 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname smithi112 2024-03-21T17:27:12.183 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/smithi046/smithi112/g' /etc/hosts 2024-03-21T17:27:12.215 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/172.21.15.46/172.21.15.112/g' /etc/hosts 2024-03-21T17:27:12.286 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/smithi046/smithi112/g' /etc/hostname 2024-03-21T17:27:12.355 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname smithi112 2024-03-21T17:27:12.425 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-03-21T17:27:12.482 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-03-21T17:27:12.482 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-03-21T17:27:12.482 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-03-21T17:27:12.482 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-03-21T17:27:12.482 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-03-21T17:27:12.482 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-03-21T17:27:12.482 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T17:27:12.482 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-03-21T17:27:12.483 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-03-21T17:27:12.483 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T17:27:12.483 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-03-21T17:27:12.483 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T17:27:12.483 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T17:27:12.483 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T17:27:12.486 INFO:teuthology.provision.fog.smithi112:Deploy complete! 2024-03-21T17:27:12.487 DEBUG:teuthology.parallel:result is None 2024-03-21T17:27:16.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:28:09.867 WARNING:teuthology.provision.fog.smithi084:timed out 2024-03-21T17:28:15.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:28:16.178 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-21T17:28:16.178 DEBUG:teuthology.orchestra.run.smithi084:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-21T17:28:16.523 INFO:teuthology.provision.fog.smithi084:Node is ready 2024-03-21T17:28:16.523 DEBUG:teuthology.orchestra.run.smithi084:> hostname 2024-03-21T17:28:16.538 INFO:teuthology.orchestra.run.smithi084.stdout:smithi084.front.sepia.ceph.com 2024-03-21T17:28:16.538 DEBUG:teuthology.orchestra.run.smithi084:> grep smithi084.front.sepia.ceph.com /etc/hosts 2024-03-21T17:28:16.593 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84 smithi084.front.sepia.ceph.com smithi084 2024-03-21T17:28:16.594 DEBUG:teuthology.orchestra.run.smithi084:> sudo hostname smithi084 2024-03-21T17:28:16.696 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i -e 's/smithi084.front.sepia.ceph.com/smithi084/g' /etc/hosts 2024-03-21T17:28:16.724 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i -e 's/172.21.15.84/172.21.15.84/g' /etc/hosts 2024-03-21T17:28:16.792 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i -e 's/smithi084.front.sepia.ceph.com/smithi084/g' /etc/hostname 2024-03-21T17:28:16.859 DEBUG:teuthology.orchestra.run.smithi084:> sudo hostname smithi084 2024-03-21T17:28:16.917 WARNING:teuthology.provision.fog.smithi088:timed out 2024-03-21T17:28:16.924 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release 2024-03-21T17:28:16.979 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="CentOS Stream" 2024-03-21T17:28:16.979 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="9" 2024-03-21T17:28:16.979 INFO:teuthology.orchestra.run.smithi084.stdout:ID="centos" 2024-03-21T17:28:16.979 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="rhel fedora" 2024-03-21T17:28:16.979 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="9" 2024-03-21T17:28:16.979 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el9" 2024-03-21T17:28:16.979 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T17:28:16.979 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2024-03-21T17:28:16.980 INFO:teuthology.orchestra.run.smithi084.stdout:LOGO="fedora-logo-icon" 2024-03-21T17:28:16.980 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T17:28:16.980 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://centos.org/" 2024-03-21T17:28:16.980 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T17:28:16.980 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T17:28:16.980 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T17:28:16.980 INFO:teuthology.provision.fog.smithi084:Deploy complete! 2024-03-21T17:28:16.981 DEBUG:teuthology.parallel:result is None 2024-03-21T17:28:22.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:28:23.171 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-21T17:28:23.171 DEBUG:teuthology.orchestra.run.smithi088:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-21T17:28:23.506 INFO:teuthology.provision.fog.smithi088:Node is ready 2024-03-21T17:28:23.506 DEBUG:teuthology.orchestra.run.smithi088:> hostname 2024-03-21T17:28:23.520 INFO:teuthology.orchestra.run.smithi088.stdout:smithi088.front.sepia.ceph.com 2024-03-21T17:28:23.521 DEBUG:teuthology.orchestra.run.smithi088:> grep smithi088.front.sepia.ceph.com /etc/hosts 2024-03-21T17:28:23.576 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88 smithi088.front.sepia.ceph.com smithi088 2024-03-21T17:28:23.577 DEBUG:teuthology.orchestra.run.smithi088:> sudo hostname smithi088 2024-03-21T17:28:23.683 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/smithi088.front.sepia.ceph.com/smithi088/g' /etc/hosts 2024-03-21T17:28:23.712 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/172.21.15.88/172.21.15.88/g' /etc/hosts 2024-03-21T17:28:23.779 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/smithi088.front.sepia.ceph.com/smithi088/g' /etc/hostname 2024-03-21T17:28:23.847 DEBUG:teuthology.orchestra.run.smithi088:> sudo hostname smithi088 2024-03-21T17:28:23.912 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2024-03-21T17:28:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Stream" 2024-03-21T17:28:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="9" 2024-03-21T17:28:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos" 2024-03-21T17:28:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora" 2024-03-21T17:28:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="9" 2024-03-21T17:28:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el9" 2024-03-21T17:28:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T17:28:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31" 2024-03-21T17:28:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:LOGO="fedora-logo-icon" 2024-03-21T17:28:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T17:28:23.968 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://centos.org/" 2024-03-21T17:28:23.968 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T17:28:23.968 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T17:28:23.968 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T17:28:23.968 INFO:teuthology.provision.fog.smithi088:Deploy complete! 2024-03-21T17:28:23.968 DEBUG:teuthology.parallel:result is None 2024-03-21T17:28:24.180 INFO:teuthology.lock.ops:Checking smithi084.front.sepia.ceph.com 2024-03-21T17:28:24.181 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-21T17:28:24.206 INFO:teuthology.lock.ops:Checking smithi088.front.sepia.ceph.com 2024-03-21T17:28:24.206 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-21T17:28:24.231 INFO:teuthology.lock.ops:Checking smithi112.front.sepia.ceph.com 2024-03-21T17:28:24.231 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-21T17:28:24.256 INFO:teuthology.lock.ops:Updating [smithi084.front.sepia.ceph.com]: set os type and version on server 2024-03-21T17:28:24.257 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-03-21T17:28:24.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:28:24.347 DEBUG:teuthology.orchestra.run.smithi084:> true 2024-03-21T17:28:24.414 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-03-21T17:28:24.415 DEBUG:teuthology.orchestra.run.smithi084:> uname -m 2024-03-21T17:28:24.470 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2024-03-21T17:28:24.470 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release 2024-03-21T17:28:24.529 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="CentOS Stream" 2024-03-21T17:28:24.529 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="9" 2024-03-21T17:28:24.529 INFO:teuthology.orchestra.run.smithi084.stdout:ID="centos" 2024-03-21T17:28:24.529 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="rhel fedora" 2024-03-21T17:28:24.529 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="9" 2024-03-21T17:28:24.529 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el9" 2024-03-21T17:28:24.530 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T17:28:24.530 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2024-03-21T17:28:24.530 INFO:teuthology.orchestra.run.smithi084.stdout:LOGO="fedora-logo-icon" 2024-03-21T17:28:24.530 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T17:28:24.530 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://centos.org/" 2024-03-21T17:28:24.530 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T17:28:24.530 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T17:28:24.530 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T17:28:24.531 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2024-03-21T17:28:24.556 INFO:teuthology.lock.ops:Updating [smithi088.front.sepia.ceph.com]: set os type and version on server 2024-03-21T17:28:24.556 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-03-21T17:28:24.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:28:24.646 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-03-21T17:28:24.716 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-03-21T17:28:24.716 DEBUG:teuthology.orchestra.run.smithi088:> uname -m 2024-03-21T17:28:24.771 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64 2024-03-21T17:28:24.771 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2024-03-21T17:28:24.825 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Stream" 2024-03-21T17:28:24.825 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="9" 2024-03-21T17:28:24.825 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos" 2024-03-21T17:28:24.826 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora" 2024-03-21T17:28:24.826 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="9" 2024-03-21T17:28:24.826 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el9" 2024-03-21T17:28:24.826 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T17:28:24.826 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31" 2024-03-21T17:28:24.826 INFO:teuthology.orchestra.run.smithi088.stdout:LOGO="fedora-logo-icon" 2024-03-21T17:28:24.826 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T17:28:24.826 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://centos.org/" 2024-03-21T17:28:24.826 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T17:28:24.826 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T17:28:24.826 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T17:28:24.827 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2024-03-21T17:28:24.847 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: set os type and version on server 2024-03-21T17:28:24.847 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-03-21T17:28:24.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T17:28:24.937 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-03-21T17:28:25.108 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-03-21T17:28:25.108 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-03-21T17:28:25.162 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-03-21T17:28:25.163 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-03-21T17:28:25.217 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-03-21T17:28:25.217 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-03-21T17:28:25.218 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-03-21T17:28:25.218 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-03-21T17:28:25.218 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-03-21T17:28:25.218 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-03-21T17:28:25.218 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T17:28:25.218 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-03-21T17:28:25.218 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-03-21T17:28:25.218 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T17:28:25.218 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-03-21T17:28:25.218 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T17:28:25.218 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T17:28:25.218 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T17:28:25.219 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-03-21T17:28:25.247 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T17:28:25.339 INFO:teuthology.dispatcher.supervisor:Running job 7613986 2024-03-21T17:28:25.339 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-03-20_21:24:03-fs-squid-distro-default-smithi/7613986 --name teuthology-2024-03-20_21:24:03-fs-squid-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/fsync-tester}} -- /home/teuthworker/archive/teuthology-2024-03-20_21:24:03-fs-squid-distro-default-smithi/7613986/orig.config.yaml 2024-03-21T17:28:25.342 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2024-03-20_21:24:03-fs-squid-distro-default-smithi/7613986 2024-03-21T17:28:25.342 INFO:teuthology.dispatcher.supervisor:Job PID: 2221835 2024-03-21T17:28:25.342 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-03-21T17:30:25.437 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T17:32:25.603 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T17:34:25.767 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T17:36:25.939 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T17:38:26.088 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T17:40:26.242 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T17:42:26.417 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T17:44:26.584 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T17:46:26.759 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T17:48:26.924 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T17:50:27.084 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T17:52:27.271 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T17:54:27.453 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T17:56:27.635 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T17:58:27.799 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T18:00:27.963 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T18:02:28.125 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T18:02:28.189 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1