2024-04-14T03:57:33.476 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-04-14T03:57:33.476 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T03:57:33.657 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f966f688c10>, signals=[15]) 2024-04-14T03:57:33.657 INFO:teuthology.lock.ops:Start node 'smithi023.front.sepia.ceph.com' reimaging 2024-04-14T03:57:33.657 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: reset os type and version on server 2024-04-14T03:57:33.657 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server 2024-04-14T03:57:33.678 INFO:teuthology.lock.ops:Start node 'smithi044.front.sepia.ceph.com' reimaging 2024-04-14T03:57:33.678 INFO:teuthology.lock.ops:Updating [smithi044.front.sepia.ceph.com]: reset os type and version on server 2024-04-14T03:57:33.678 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-04-14T03:57:33.702 INFO:teuthology.lock.ops:Start node 'smithi098.front.sepia.ceph.com' reimaging 2024-04-14T03:57:33.703 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: reset os type and version on server 2024-04-14T03:57:33.703 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2024-04-14T03:57:33.809 DEBUG:teuthology.provision.fog.smithi023:Requesting image smithi_centos_9.stream (ID 10) 2024-04-14T03:57:33.828 DEBUG:teuthology.provision.fog.smithi044:Requesting image smithi_centos_9.stream (ID 10) 2024-04-14T03:57:33.834 DEBUG:teuthology.provision.fog.smithi098:Requesting image smithi_centos_9.stream (ID 10) 2024-04-14T03:57:33.852 INFO:teuthology.provision.fog.smithi023:Scheduling deploy of centos 9.stream 2024-04-14T03:57:33.871 INFO:teuthology.provision.fog.smithi044:Scheduling deploy of centos 9.stream 2024-04-14T03:57:33.878 INFO:teuthology.provision.fog.smithi098:Scheduling deploy of centos 9.stream 2024-04-14T03:57:34.063 INFO:teuthology.orchestra.console.smithi023:Power off 2024-04-14T03:57:34.064 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-14T03:57:34.087 DEBUG:teuthology.orchestra.console.smithi023:power off output: Chassis Power Control: Down/Off 2024-04-14T03:57:34.087 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:34.108 INFO:teuthology.orchestra.console.smithi098:Power off 2024-04-14T03:57:34.108 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-14T03:57:34.121 INFO:teuthology.orchestra.console.smithi044:Power off 2024-04-14T03:57:34.121 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-14T03:57:34.129 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on 2024-04-14T03:57:34.130 DEBUG:teuthology.orchestra.console.smithi098:power off output: Chassis Power Control: Down/Off 2024-04-14T03:57:34.131 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:34.143 DEBUG:teuthology.orchestra.console.smithi044:power off output: Chassis Power Control: Down/Off 2024-04-14T03:57:34.143 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:34.152 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on 2024-04-14T03:57:34.165 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-04-14T03:57:38.134 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:38.153 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:38.165 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:38.249 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on 2024-04-14T03:57:38.262 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on 2024-04-14T03:57:38.275 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-04-14T03:57:42.254 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:42.263 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:42.276 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:42.364 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on 2024-04-14T03:57:42.372 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off 2024-04-14T03:57:42.385 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-04-14T03:57:42.473 INFO:teuthology.orchestra.console.smithi098:Power off completed 2024-04-14T03:57:42.574 INFO:teuthology.orchestra.console.smithi098:Power on 2024-04-14T03:57:42.574 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-14T03:57:42.597 DEBUG:teuthology.orchestra.console.smithi098:power on output: Chassis Power Control: Up/On 2024-04-14T03:57:42.598 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:42.621 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off 2024-04-14T03:57:46.367 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:46.386 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:46.476 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off 2024-04-14T03:57:46.495 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2024-04-14T03:57:46.577 INFO:teuthology.orchestra.console.smithi023:Power off completed 2024-04-14T03:57:46.596 INFO:teuthology.orchestra.console.smithi044:Power off completed 2024-04-14T03:57:46.623 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:46.678 INFO:teuthology.orchestra.console.smithi023:Power on 2024-04-14T03:57:46.678 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-14T03:57:46.697 INFO:teuthology.orchestra.console.smithi044:Power on 2024-04-14T03:57:46.697 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-14T03:57:46.705 DEBUG:teuthology.orchestra.console.smithi023:power on output: Chassis Power Control: Up/On 2024-04-14T03:57:46.705 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:46.718 DEBUG:teuthology.orchestra.console.smithi044:power on output: Chassis Power Control: Up/On 2024-04-14T03:57:46.718 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:46.730 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off 2024-04-14T03:57:46.733 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off 2024-04-14T03:57:46.742 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2024-04-14T03:57:50.734 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:50.741 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:50.749 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:50.843 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on 2024-04-14T03:57:50.857 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2024-04-14T03:57:50.865 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on 2024-04-14T03:57:50.944 INFO:teuthology.orchestra.console.smithi023:Power on completed 2024-04-14T03:57:50.966 INFO:teuthology.orchestra.console.smithi098:Power on completed 2024-04-14T03:57:51.045 INFO:teuthology.provision.fog.smithi023:Waiting for deploy to finish 2024-04-14T03:57:51.067 INFO:teuthology.provision.fog.smithi098:Waiting for deploy to finish 2024-04-14T03:57:54.861 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T03:57:54.970 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-04-14T03:57:55.071 INFO:teuthology.orchestra.console.smithi044:Power on completed 2024-04-14T03:57:55.172 INFO:teuthology.provision.fog.smithi044:Waiting for deploy to finish 2024-04-14T04:00:21.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:00:24.862 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2024-04-14T04:00:25.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:00:28.862 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2024-04-14T04:00:30.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:00:33.922 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2024-04-14T04:00:34.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:00:37.918 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2024-04-14T04:00:39.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:00:43.006 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2024-04-14T04:00:43.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:00:46.974 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2024-04-14T04:00:49.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:00:52.066 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2024-04-14T04:00:52.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:00:56.034 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2024-04-14T04:00:58.069 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:01:01.150 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2024-04-14T04:01:02.037 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:01:05.118 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2024-04-14T04:01:06.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:01:07.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:01:10.046 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T04:01:10.206 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2024-04-14T04:01:11.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:01:16.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:01:16.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:01:19.102 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T04:01:25.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:01:28.158 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T04:01:34.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:01:37.214 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T04:01:43.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:01:46.274 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T04:01:52.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:01:55.358 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T04:02:01.361 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:02:04.414 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T04:02:10.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:02:11.125 WARNING:teuthology.provision.fog.smithi044:timed out 2024-04-14T04:02:13.474 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T04:02:16.210 WARNING:teuthology.provision.fog.smithi098:timed out 2024-04-14T04:02:17.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:02:17.385 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-14T04:02:17.385 DEBUG:teuthology.orchestra.run.smithi044:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-14T04:02:17.746 INFO:teuthology.provision.fog.smithi044:Node is ready 2024-04-14T04:02:17.746 DEBUG:teuthology.orchestra.run.smithi044:> hostname 2024-04-14T04:02:17.762 INFO:teuthology.orchestra.run.smithi044.stdout:smithi044.front.sepia.ceph.com 2024-04-14T04:02:17.762 DEBUG:teuthology.orchestra.run.smithi044:> grep smithi044.front.sepia.ceph.com /etc/hosts 2024-04-14T04:02:17.817 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44 smithi044.front.sepia.ceph.com smithi044 2024-04-14T04:02:17.818 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname smithi044 2024-04-14T04:02:17.931 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/smithi044.front.sepia.ceph.com/smithi044/g' /etc/hosts 2024-04-14T04:02:17.963 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/172.21.15.44/172.21.15.44/g' /etc/hosts 2024-04-14T04:02:18.034 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/smithi044.front.sepia.ceph.com/smithi044/g' /etc/hostname 2024-04-14T04:02:18.105 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname smithi044 2024-04-14T04:02:18.175 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-04-14T04:02:18.230 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2024-04-14T04:02:18.230 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="9" 2024-04-14T04:02:18.230 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2024-04-14T04:02:18.230 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2024-04-14T04:02:18.230 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="9" 2024-04-14T04:02:18.230 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el9" 2024-04-14T04:02:18.230 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T04:02:18.230 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-04-14T04:02:18.230 INFO:teuthology.orchestra.run.smithi044.stdout:LOGO="fedora-logo-icon" 2024-04-14T04:02:18.231 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T04:02:18.231 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2024-04-14T04:02:18.231 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T04:02:18.231 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T04:02:18.231 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T04:02:18.232 INFO:teuthology.provision.fog.smithi044:Deploy complete! 2024-04-14T04:02:18.233 DEBUG:teuthology.parallel:result is None 2024-04-14T04:02:19.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:02:22.215 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:02:22.462 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-14T04:02:22.463 DEBUG:teuthology.orchestra.run.smithi098:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-14T04:02:22.558 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T04:02:22.773 INFO:teuthology.provision.fog.smithi098:Node is ready 2024-04-14T04:02:22.774 DEBUG:teuthology.orchestra.run.smithi098:> hostname 2024-04-14T04:02:22.788 INFO:teuthology.orchestra.run.smithi098.stdout:smithi098.front.sepia.ceph.com 2024-04-14T04:02:22.788 DEBUG:teuthology.orchestra.run.smithi098:> grep smithi098.front.sepia.ceph.com /etc/hosts 2024-04-14T04:02:22.843 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98 smithi098.front.sepia.ceph.com smithi098 2024-04-14T04:02:22.844 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname smithi098 2024-04-14T04:02:22.957 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/smithi098.front.sepia.ceph.com/smithi098/g' /etc/hosts 2024-04-14T04:02:22.985 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/172.21.15.98/172.21.15.98/g' /etc/hosts 2024-04-14T04:02:23.053 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/smithi098.front.sepia.ceph.com/smithi098/g' /etc/hostname 2024-04-14T04:02:23.121 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname smithi098 2024-04-14T04:02:23.187 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2024-04-14T04:02:23.242 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream" 2024-04-14T04:02:23.242 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="9" 2024-04-14T04:02:23.242 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2024-04-14T04:02:23.242 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2024-04-14T04:02:23.242 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="9" 2024-04-14T04:02:23.242 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el9" 2024-04-14T04:02:23.242 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T04:02:23.242 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2024-04-14T04:02:23.242 INFO:teuthology.orchestra.run.smithi098.stdout:LOGO="fedora-logo-icon" 2024-04-14T04:02:23.242 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T04:02:23.243 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/" 2024-04-14T04:02:23.243 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T04:02:23.243 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T04:02:23.243 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T04:02:23.243 INFO:teuthology.provision.fog.smithi098:Deploy complete! 2024-04-14T04:02:23.244 DEBUG:teuthology.parallel:result is None 2024-04-14T04:02:28.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:02:31.614 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T04:02:37.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:03:37.683 WARNING:teuthology.provision.fog.smithi023:timed out 2024-04-14T04:03:43.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:03:43.961 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-14T04:03:43.962 DEBUG:teuthology.orchestra.run.smithi023:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-14T04:03:44.288 INFO:teuthology.provision.fog.smithi023:Node is ready 2024-04-14T04:03:44.288 DEBUG:teuthology.orchestra.run.smithi023:> hostname 2024-04-14T04:03:44.303 INFO:teuthology.orchestra.run.smithi023.stdout:smithi023.front.sepia.ceph.com 2024-04-14T04:03:44.303 DEBUG:teuthology.orchestra.run.smithi023:> grep smithi023.front.sepia.ceph.com /etc/hosts 2024-04-14T04:03:44.359 INFO:teuthology.orchestra.run.smithi023.stdout:172.21.15.23 smithi023.front.sepia.ceph.com smithi023 2024-04-14T04:03:44.359 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023 2024-04-14T04:03:44.472 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hosts 2024-04-14T04:03:44.503 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/172.21.15.23/172.21.15.23/g' /etc/hosts 2024-04-14T04:03:44.573 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hostname 2024-04-14T04:03:44.645 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023 2024-04-14T04:03:44.715 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release 2024-04-14T04:03:44.770 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Stream" 2024-04-14T04:03:44.770 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="9" 2024-04-14T04:03:44.770 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos" 2024-04-14T04:03:44.770 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora" 2024-04-14T04:03:44.770 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="9" 2024-04-14T04:03:44.770 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el9" 2024-04-14T04:03:44.770 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T04:03:44.770 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31" 2024-04-14T04:03:44.770 INFO:teuthology.orchestra.run.smithi023.stdout:LOGO="fedora-logo-icon" 2024-04-14T04:03:44.771 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T04:03:44.771 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://centos.org/" 2024-04-14T04:03:44.771 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T04:03:44.771 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T04:03:44.771 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T04:03:44.771 INFO:teuthology.provision.fog.smithi023:Deploy complete! 2024-04-14T04:03:44.772 DEBUG:teuthology.parallel:result is None 2024-04-14T04:03:44.996 INFO:teuthology.lock.ops:Checking smithi023.front.sepia.ceph.com 2024-04-14T04:03:44.996 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-14T04:03:45.023 INFO:teuthology.lock.ops:Checking smithi044.front.sepia.ceph.com 2024-04-14T04:03:45.023 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-14T04:03:45.039 INFO:teuthology.lock.ops:Checking smithi098.front.sepia.ceph.com 2024-04-14T04:03:45.040 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-14T04:03:45.060 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: set os type and version on server 2024-04-14T04:03:45.061 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2024-04-14T04:03:45.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:03:45.168 DEBUG:teuthology.orchestra.run.smithi023:> true 2024-04-14T04:03:45.233 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi023.front.sepia.ceph.com' 2024-04-14T04:03:45.233 DEBUG:teuthology.orchestra.run.smithi023:> uname -m 2024-04-14T04:03:45.287 INFO:teuthology.orchestra.run.smithi023.stdout:x86_64 2024-04-14T04:03:45.288 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release 2024-04-14T04:03:45.342 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Stream" 2024-04-14T04:03:45.343 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="9" 2024-04-14T04:03:45.343 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos" 2024-04-14T04:03:45.343 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora" 2024-04-14T04:03:45.343 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="9" 2024-04-14T04:03:45.343 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el9" 2024-04-14T04:03:45.343 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T04:03:45.343 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31" 2024-04-14T04:03:45.343 INFO:teuthology.orchestra.run.smithi023.stdout:LOGO="fedora-logo-icon" 2024-04-14T04:03:45.343 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T04:03:45.343 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://centos.org/" 2024-04-14T04:03:45.343 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T04:03:45.343 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T04:03:45.343 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T04:03:45.344 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server 2024-04-14T04:03:45.364 INFO:teuthology.lock.ops:Updating [smithi044.front.sepia.ceph.com]: set os type and version on server 2024-04-14T04:03:45.364 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-14T04:03:45.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:03:45.468 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-04-14T04:03:45.653 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-14T04:03:45.653 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2024-04-14T04:03:45.708 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2024-04-14T04:03:45.708 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-04-14T04:03:45.762 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2024-04-14T04:03:45.763 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="9" 2024-04-14T04:03:45.763 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2024-04-14T04:03:45.763 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2024-04-14T04:03:45.763 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="9" 2024-04-14T04:03:45.764 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el9" 2024-04-14T04:03:45.764 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T04:03:45.764 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-04-14T04:03:45.764 INFO:teuthology.orchestra.run.smithi044.stdout:LOGO="fedora-logo-icon" 2024-04-14T04:03:45.764 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T04:03:45.764 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2024-04-14T04:03:45.764 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T04:03:45.764 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T04:03:45.764 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T04:03:45.764 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-04-14T04:03:45.790 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: set os type and version on server 2024-04-14T04:03:45.790 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-04-14T04:03:45.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T04:03:45.878 DEBUG:teuthology.orchestra.run.smithi098:> true 2024-04-14T04:03:46.049 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-04-14T04:03:46.049 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2024-04-14T04:03:46.103 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2024-04-14T04:03:46.104 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2024-04-14T04:03:46.158 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream" 2024-04-14T04:03:46.159 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="9" 2024-04-14T04:03:46.159 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2024-04-14T04:03:46.159 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2024-04-14T04:03:46.159 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="9" 2024-04-14T04:03:46.159 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el9" 2024-04-14T04:03:46.159 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T04:03:46.159 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2024-04-14T04:03:46.159 INFO:teuthology.orchestra.run.smithi098.stdout:LOGO="fedora-logo-icon" 2024-04-14T04:03:46.159 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T04:03:46.159 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/" 2024-04-14T04:03:46.159 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T04:03:46.159 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T04:03:46.159 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T04:03:46.160 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2024-04-14T04:03:46.181 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T04:03:46.269 INFO:teuthology.dispatcher.supervisor:Running job 7637002 2024-04-14T04:03:46.269 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-02_20:16:13-rbd-main-distro-default-smithi/7637002 --name teuthology-2024-04-02_20:16:13-rbd-main-distro-default-smithi --description rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/diff-continuous-krbd} -- /home/teuthworker/archive/teuthology-2024-04-02_20:16:13-rbd-main-distro-default-smithi/7637002/orig.config.yaml 2024-04-14T04:03:46.272 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2024-04-02_20:16:13-rbd-main-distro-default-smithi/7637002 2024-04-14T04:03:46.273 INFO:teuthology.dispatcher.supervisor:Job PID: 640610 2024-04-14T04:03:46.273 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-14T04:05:46.371 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T04:07:46.537 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T04:09:46.711 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T04:11:46.873 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T04:13:47.040 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T04:15:47.211 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T04:17:47.383 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T04:19:47.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T04:21:47.719 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T04:23:47.859 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T04:25:48.027 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T04:27:48.193 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T04:27:48.270 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1