2024-03-13T22:16:44.373 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-13T22:16:44.374 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T22:16:44.556 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f249b135c10>, signals=[15]) 2024-03-13T22:16:44.556 INFO:teuthology.lock.ops:Start node 'smithi089.front.sepia.ceph.com' reimaging 2024-03-13T22:16:44.556 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: reset os type and version on server 2024-03-13T22:16:44.557 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2024-03-13T22:16:44.573 INFO:teuthology.lock.ops:Start node 'smithi112.front.sepia.ceph.com' reimaging 2024-03-13T22:16:44.574 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: reset os type and version on server 2024-03-13T22:16:44.574 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-03-13T22:16:44.598 INFO:teuthology.lock.ops:Start node 'smithi177.front.sepia.ceph.com' reimaging 2024-03-13T22:16:44.599 INFO:teuthology.lock.ops:Updating [smithi177.front.sepia.ceph.com]: reset os type and version on server 2024-03-13T22:16:44.599 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2024-03-13T22:16:44.693 DEBUG:teuthology.provision.fog.smithi089:Requesting image smithi_centos_9.stream (ID 10) 2024-03-13T22:16:44.729 DEBUG:teuthology.provision.fog.smithi112:Requesting image smithi_centos_9.stream (ID 10) 2024-03-13T22:16:44.741 DEBUG:teuthology.provision.fog.smithi177:Requesting image smithi_centos_9.stream (ID 10) 2024-03-13T22:16:44.745 INFO:teuthology.provision.fog.smithi089:Scheduling deploy of centos 9.stream 2024-03-13T22:16:44.779 INFO:teuthology.provision.fog.smithi112:Scheduling deploy of centos 9.stream 2024-03-13T22:16:44.792 INFO:teuthology.provision.fog.smithi177:Scheduling deploy of centos 9.stream 2024-03-13T22:16:45.040 INFO:teuthology.orchestra.console.smithi089:Power off 2024-03-13T22:16:45.040 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-13T22:16:45.052 INFO:teuthology.orchestra.console.smithi112:Power off 2024-03-13T22:16:45.052 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-13T22:16:45.069 DEBUG:teuthology.orchestra.console.smithi089:power off output: Chassis Power Control: Down/Off 2024-03-13T22:16:45.069 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T22:16:45.079 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off 2024-03-13T22:16:45.079 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T22:16:45.091 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2024-03-13T22:16:45.100 INFO:teuthology.orchestra.console.smithi177:Power off 2024-03-13T22:16:45.100 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-13T22:16:45.109 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-03-13T22:16:45.121 DEBUG:teuthology.orchestra.console.smithi177:power off output: Chassis Power Control: Down/Off 2024-03-13T22:16:45.121 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T22:16:45.144 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2024-03-13T22:16:45.210 INFO:teuthology.orchestra.console.smithi112:Power off completed 2024-03-13T22:16:45.311 INFO:teuthology.orchestra.console.smithi112:Power on 2024-03-13T22:16:45.311 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-13T22:16:45.333 DEBUG:teuthology.orchestra.console.smithi112:power on output: Chassis Power Control: Up/On 2024-03-13T22:16:45.334 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T22:16:45.357 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-03-13T22:16:49.094 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T22:16:49.145 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T22:16:49.203 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2024-03-13T22:16:49.256 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2024-03-13T22:16:49.358 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T22:16:49.467 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-03-13T22:16:49.568 INFO:teuthology.orchestra.console.smithi112:Power on completed 2024-03-13T22:16:49.669 INFO:teuthology.provision.fog.smithi112:Waiting for deploy to finish 2024-03-13T22:16:53.206 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T22:16:53.257 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T22:16:53.315 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off 2024-03-13T22:16:53.366 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2024-03-13T22:16:53.417 INFO:teuthology.orchestra.console.smithi089:Power off completed 2024-03-13T22:16:53.517 INFO:teuthology.orchestra.console.smithi089:Power on 2024-03-13T22:16:53.517 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-13T22:16:53.539 DEBUG:teuthology.orchestra.console.smithi089:power on output: Chassis Power Control: Up/On 2024-03-13T22:16:53.539 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T22:16:53.560 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off 2024-03-13T22:16:57.370 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T22:16:57.479 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off 2024-03-13T22:16:57.561 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T22:16:57.580 INFO:teuthology.orchestra.console.smithi177:Power off completed 2024-03-13T22:16:57.671 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off 2024-03-13T22:16:57.681 INFO:teuthology.orchestra.console.smithi177:Power on 2024-03-13T22:16:57.681 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-13T22:16:57.701 DEBUG:teuthology.orchestra.console.smithi177:power on output: Chassis Power Control: Up/On 2024-03-13T22:16:57.701 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T22:16:57.724 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off 2024-03-13T22:17:01.676 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T22:17:01.724 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T22:17:01.785 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2024-03-13T22:17:01.834 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2024-03-13T22:17:01.887 INFO:teuthology.orchestra.console.smithi089:Power on completed 2024-03-13T22:17:01.935 INFO:teuthology.orchestra.console.smithi177:Power on completed 2024-03-13T22:17:01.988 INFO:teuthology.provision.fog.smithi089:Waiting for deploy to finish 2024-03-13T22:17:02.036 INFO:teuthology.provision.fog.smithi177:Waiting for deploy to finish 2024-03-13T22:19:20.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:19:23.774 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-03-13T22:19:29.779 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:19:32.830 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-03-13T22:19:33.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:19:33.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:19:33.390 ERROR:teuthology.orchestra.connection:Error authenticating with smithi177.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-03-13T22:19:36.162 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-03-13T22:19:38.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:19:41.886 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-03-13T22:19:42.164 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:19:45.218 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-03-13T22:19:47.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:19:50.942 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-03-13T22:19:51.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:19:54.270 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-03-13T22:19:56.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:19:59.998 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-03-13T22:20:00.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:20:03.326 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-03-13T22:20:06.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:20:09.054 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-03-13T22:20:09.327 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:20:12.382 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-03-13T22:20:15.056 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:20:18.110 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-03-13T22:20:18.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:20:21.438 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-03-13T22:20:24.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:20:27.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:20:37.393 WARNING:teuthology.provision.fog.smithi177:timed out 2024-03-13T22:20:43.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:20:46.466 WARNING:teuthology.provision.fog.smithi177:[Errno None] Unable to connect to port 22 on 172.21.15.177 2024-03-13T22:20:52.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:20:55.518 WARNING:teuthology.provision.fog.smithi177:[Errno None] Unable to connect to port 22 on 172.21.15.177 2024-03-13T22:21:01.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:21:04.574 WARNING:teuthology.provision.fog.smithi177:[Errno None] Unable to connect to port 22 on 172.21.15.177 2024-03-13T22:21:10.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:21:13.662 WARNING:teuthology.provision.fog.smithi177:[Errno None] Unable to connect to port 22 on 172.21.15.177 2024-03-13T22:21:19.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:21:19.874 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-13T22:21:19.874 DEBUG:teuthology.orchestra.run.smithi177:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-13T22:21:24.113 WARNING:teuthology.provision.fog.smithi112:timed out 2024-03-13T22:21:27.442 WARNING:teuthology.provision.fog.smithi089:timed out 2024-03-13T22:21:30.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:21:30.438 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-13T22:21:30.438 DEBUG:teuthology.orchestra.run.smithi112:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-13T22:21:30.771 INFO:teuthology.provision.fog.smithi112:Node is ready 2024-03-13T22:21:30.771 DEBUG:teuthology.orchestra.run.smithi112:> hostname 2024-03-13T22:21:30.785 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112.front.sepia.ceph.com 2024-03-13T22:21:30.786 DEBUG:teuthology.orchestra.run.smithi112:> grep smithi112.front.sepia.ceph.com /etc/hosts 2024-03-13T22:21:30.841 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112 smithi112.front.sepia.ceph.com smithi112 2024-03-13T22:21:30.841 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname smithi112 2024-03-13T22:21:30.941 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/smithi112.front.sepia.ceph.com/smithi112/g' /etc/hosts 2024-03-13T22:21:30.968 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/172.21.15.112/172.21.15.112/g' /etc/hosts 2024-03-13T22:21:31.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/smithi112.front.sepia.ceph.com/smithi112/g' /etc/hostname 2024-03-13T22:21:31.105 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname smithi112 2024-03-13T22:21:31.171 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-03-13T22:21:31.225 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-03-13T22:21:31.225 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-03-13T22:21:31.225 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-03-13T22:21:31.225 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-03-13T22:21:31.226 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-03-13T22:21:31.226 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-03-13T22:21:31.226 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-13T22:21:31.226 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-03-13T22:21:31.226 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-03-13T22:21:31.226 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-13T22:21:31.226 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-03-13T22:21:31.226 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T22:21:31.226 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-13T22:21:31.226 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-13T22:21:31.227 INFO:teuthology.provision.fog.smithi112:Deploy complete! 2024-03-13T22:21:31.228 DEBUG:teuthology.parallel:result is None 2024-03-13T22:21:33.443 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:21:33.806 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-13T22:21:33.807 DEBUG:teuthology.orchestra.run.smithi089:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-13T22:21:34.140 INFO:teuthology.provision.fog.smithi089:Node is ready 2024-03-13T22:21:34.140 DEBUG:teuthology.orchestra.run.smithi089:> hostname 2024-03-13T22:21:34.154 INFO:teuthology.orchestra.run.smithi089.stdout:smithi089.front.sepia.ceph.com 2024-03-13T22:21:34.154 DEBUG:teuthology.orchestra.run.smithi089:> grep smithi089.front.sepia.ceph.com /etc/hosts 2024-03-13T22:21:34.209 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89 smithi089.front.sepia.ceph.com smithi089 2024-03-13T22:21:34.210 DEBUG:teuthology.orchestra.run.smithi089:> sudo hostname smithi089 2024-03-13T22:21:34.321 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i -e 's/smithi089.front.sepia.ceph.com/smithi089/g' /etc/hosts 2024-03-13T22:21:34.349 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i -e 's/172.21.15.89/172.21.15.89/g' /etc/hosts 2024-03-13T22:21:34.415 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i -e 's/smithi089.front.sepia.ceph.com/smithi089/g' /etc/hostname 2024-03-13T22:21:34.482 DEBUG:teuthology.orchestra.run.smithi089:> sudo hostname smithi089 2024-03-13T22:21:34.549 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2024-03-13T22:21:34.603 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream" 2024-03-13T22:21:34.604 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="9" 2024-03-13T22:21:34.604 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos" 2024-03-13T22:21:34.604 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora" 2024-03-13T22:21:34.604 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="9" 2024-03-13T22:21:34.604 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el9" 2024-03-13T22:21:34.605 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-13T22:21:34.605 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2024-03-13T22:21:34.605 INFO:teuthology.orchestra.run.smithi089.stdout:LOGO="fedora-logo-icon" 2024-03-13T22:21:34.605 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-13T22:21:34.605 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/" 2024-03-13T22:21:34.605 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T22:21:34.605 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-13T22:21:34.605 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-13T22:21:34.606 INFO:teuthology.provision.fog.smithi089:Deploy complete! 2024-03-13T22:21:34.606 DEBUG:teuthology.parallel:result is None 2024-03-13T22:21:35.209 INFO:teuthology.provision.fog.smithi177:Node is ready 2024-03-13T22:21:35.209 DEBUG:teuthology.orchestra.run.smithi177:> hostname 2024-03-13T22:21:35.226 INFO:teuthology.orchestra.run.smithi177.stdout:smithi046 2024-03-13T22:21:35.227 DEBUG:teuthology.orchestra.run.smithi177:> grep smithi046 /etc/hosts 2024-03-13T22:21:35.283 INFO:teuthology.orchestra.run.smithi177.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2024-03-13T22:21:35.284 DEBUG:teuthology.orchestra.run.smithi177:> sudo hostname smithi177 2024-03-13T22:21:35.444 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i -e 's/smithi046/smithi177/g' /etc/hosts 2024-03-13T22:21:35.475 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i -e 's/172.21.15.46/172.21.15.177/g' /etc/hosts 2024-03-13T22:21:35.544 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i -e 's/smithi046/smithi177/g' /etc/hostname 2024-03-13T22:21:35.612 DEBUG:teuthology.orchestra.run.smithi177:> sudo hostname smithi177 2024-03-13T22:21:35.680 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2024-03-13T22:21:35.734 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="CentOS Stream" 2024-03-13T22:21:35.735 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="9" 2024-03-13T22:21:35.735 INFO:teuthology.orchestra.run.smithi177.stdout:ID="centos" 2024-03-13T22:21:35.735 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="rhel fedora" 2024-03-13T22:21:35.735 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="9" 2024-03-13T22:21:35.735 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el9" 2024-03-13T22:21:35.735 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-13T22:21:35.735 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2024-03-13T22:21:35.735 INFO:teuthology.orchestra.run.smithi177.stdout:LOGO="fedora-logo-icon" 2024-03-13T22:21:35.735 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-13T22:21:35.735 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://centos.org/" 2024-03-13T22:21:35.735 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T22:21:35.735 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-13T22:21:35.735 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-13T22:21:35.736 INFO:teuthology.provision.fog.smithi177:Deploy complete! 2024-03-13T22:21:35.736 DEBUG:teuthology.parallel:result is None 2024-03-13T22:21:35.984 INFO:teuthology.lock.ops:Checking smithi089.front.sepia.ceph.com 2024-03-13T22:21:35.984 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-13T22:21:36.009 INFO:teuthology.lock.ops:Checking smithi112.front.sepia.ceph.com 2024-03-13T22:21:36.009 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-13T22:21:36.025 INFO:teuthology.lock.ops:Checking smithi177.front.sepia.ceph.com 2024-03-13T22:21:36.025 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-13T22:21:36.043 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: set os type and version on server 2024-03-13T22:21:36.043 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-03-13T22:21:36.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:21:36.131 DEBUG:teuthology.orchestra.run.smithi089:> true 2024-03-13T22:21:36.203 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-03-13T22:21:36.203 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2024-03-13T22:21:36.261 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2024-03-13T22:21:36.262 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2024-03-13T22:21:36.319 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream" 2024-03-13T22:21:36.319 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="9" 2024-03-13T22:21:36.320 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos" 2024-03-13T22:21:36.320 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora" 2024-03-13T22:21:36.320 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="9" 2024-03-13T22:21:36.320 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el9" 2024-03-13T22:21:36.320 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-13T22:21:36.320 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2024-03-13T22:21:36.320 INFO:teuthology.orchestra.run.smithi089.stdout:LOGO="fedora-logo-icon" 2024-03-13T22:21:36.320 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-13T22:21:36.320 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/" 2024-03-13T22:21:36.320 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T22:21:36.320 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-13T22:21:36.320 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-13T22:21:36.321 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2024-03-13T22:21:36.351 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: set os type and version on server 2024-03-13T22:21:36.351 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-03-13T22:21:36.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:21:36.452 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-03-13T22:21:36.523 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-03-13T22:21:36.524 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-03-13T22:21:36.580 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-03-13T22:21:36.581 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-03-13T22:21:36.638 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-03-13T22:21:36.639 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-03-13T22:21:36.639 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-03-13T22:21:36.639 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-03-13T22:21:36.639 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-03-13T22:21:36.639 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-03-13T22:21:36.639 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-13T22:21:36.639 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-03-13T22:21:36.639 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-03-13T22:21:36.639 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-13T22:21:36.639 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-03-13T22:21:36.639 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T22:21:36.639 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-13T22:21:36.639 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-13T22:21:36.640 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-03-13T22:21:36.659 INFO:teuthology.lock.ops:Updating [smithi177.front.sepia.ceph.com]: set os type and version on server 2024-03-13T22:21:36.659 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-13T22:21:36.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T22:21:36.745 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-03-13T22:21:36.817 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-03-13T22:21:36.817 DEBUG:teuthology.orchestra.run.smithi177:> uname -m 2024-03-13T22:21:36.872 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64 2024-03-13T22:21:36.873 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2024-03-13T22:21:36.927 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="CentOS Stream" 2024-03-13T22:21:36.927 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="9" 2024-03-13T22:21:36.928 INFO:teuthology.orchestra.run.smithi177.stdout:ID="centos" 2024-03-13T22:21:36.928 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="rhel fedora" 2024-03-13T22:21:36.928 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="9" 2024-03-13T22:21:36.928 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el9" 2024-03-13T22:21:36.928 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-13T22:21:36.928 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2024-03-13T22:21:36.928 INFO:teuthology.orchestra.run.smithi177.stdout:LOGO="fedora-logo-icon" 2024-03-13T22:21:36.928 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-13T22:21:36.928 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://centos.org/" 2024-03-13T22:21:36.928 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T22:21:36.928 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-13T22:21:36.928 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-13T22:21:36.929 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2024-03-13T22:21:36.951 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T22:21:37.059 INFO:teuthology.dispatcher.supervisor:Running job 7597922 2024-03-13T22:21:37.060 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-03-13_19:24:10-rados-wip-yuri3-testing-2024-03-12-0851-distro-default-smithi/7597922 --name yuriw-2024-03-13_19:24:10-rados-wip-yuri3-testing-2024-03-12-0851-distro-default-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/archive/yuriw-2024-03-13_19:24:10-rados-wip-yuri3-testing-2024-03-12-0851-distro-default-smithi/7597922/orig.config.yaml 2024-03-13T22:21:37.063 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2024-03-13_19:24:10-rados-wip-yuri3-testing-2024-03-12-0851-distro-default-smithi/7597922 2024-03-13T22:21:37.063 INFO:teuthology.dispatcher.supervisor:Job PID: 449145 2024-03-13T22:21:37.063 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-03-13T22:23:37.065 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T22:25:37.137 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T22:27:37.220 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T22:29:37.287 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T22:31:37.356 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T22:33:37.437 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T22:35:37.507 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T22:37:37.571 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T22:39:37.636 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T22:41:37.713 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T22:41:37.792 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1