2024-03-26T06:08:13.036 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-26T06:08:13.036 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:08:13.205 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3c01843c10>, signals=[15]) 2024-03-26T06:08:13.206 INFO:teuthology.lock.ops:Start node 'smithi038.front.sepia.ceph.com' reimaging 2024-03-26T06:08:13.206 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: reset os type and version on server 2024-03-26T06:08:13.206 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2024-03-26T06:08:13.227 INFO:teuthology.lock.ops:Start node 'smithi121.front.sepia.ceph.com' reimaging 2024-03-26T06:08:13.227 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: reset os type and version on server 2024-03-26T06:08:13.228 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2024-03-26T06:08:13.252 INFO:teuthology.lock.ops:Start node 'smithi170.front.sepia.ceph.com' reimaging 2024-03-26T06:08:13.252 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: reset os type and version on server 2024-03-26T06:08:13.252 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-03-26T06:08:13.345 DEBUG:teuthology.provision.fog.smithi038:Requesting image smithi_centos_9.stream (ID 10) 2024-03-26T06:08:13.355 DEBUG:teuthology.provision.fog.smithi121:Requesting image smithi_centos_9.stream (ID 10) 2024-03-26T06:08:13.379 DEBUG:teuthology.provision.fog.smithi170:Requesting image smithi_centos_9.stream (ID 10) 2024-03-26T06:08:13.390 INFO:teuthology.provision.fog.smithi038:Scheduling deploy of centos 9.stream 2024-03-26T06:08:13.399 INFO:teuthology.provision.fog.smithi121:Scheduling deploy of centos 9.stream 2024-03-26T06:08:13.424 INFO:teuthology.provision.fog.smithi170:Scheduling deploy of centos 9.stream 2024-03-26T06:08:13.617 INFO:teuthology.orchestra.console.smithi038:Power off 2024-03-26T06:08:13.618 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T06:08:13.631 INFO:teuthology.orchestra.console.smithi121:Power off 2024-03-26T06:08:13.632 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T06:08:13.646 DEBUG:teuthology.orchestra.console.smithi038:power off output: Chassis Power Control: Down/Off 2024-03-26T06:08:13.646 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:13.656 DEBUG:teuthology.orchestra.console.smithi121:power off output: Chassis Power Control: Down/Off 2024-03-26T06:08:13.656 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:13.671 INFO:teuthology.orchestra.console.smithi170:Power off 2024-03-26T06:08:13.671 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T06:08:13.681 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on 2024-03-26T06:08:13.697 DEBUG:teuthology.orchestra.console.smithi170:power off output: Chassis Power Control: Down/Off 2024-03-26T06:08:13.697 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:13.710 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is on 2024-03-26T06:08:13.719 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2024-03-26T06:08:17.686 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:17.711 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:17.720 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:17.801 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on 2024-03-26T06:08:17.820 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is on 2024-03-26T06:08:17.828 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2024-03-26T06:08:21.802 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:21.821 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:21.829 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:21.912 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off 2024-03-26T06:08:21.930 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is off 2024-03-26T06:08:21.937 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2024-03-26T06:08:22.014 INFO:teuthology.orchestra.console.smithi038:Power off completed 2024-03-26T06:08:22.030 INFO:teuthology.orchestra.console.smithi121:Power off completed 2024-03-26T06:08:22.038 INFO:teuthology.orchestra.console.smithi170:Power off completed 2024-03-26T06:08:22.115 INFO:teuthology.orchestra.console.smithi038:Power on 2024-03-26T06:08:22.116 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-26T06:08:22.131 INFO:teuthology.orchestra.console.smithi121:Power on 2024-03-26T06:08:22.132 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-26T06:08:22.140 INFO:teuthology.orchestra.console.smithi170:Power on 2024-03-26T06:08:22.141 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-26T06:08:22.149 DEBUG:teuthology.orchestra.console.smithi038:power on output: Chassis Power Control: Up/On 2024-03-26T06:08:22.150 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:22.158 DEBUG:teuthology.orchestra.console.smithi121:power on output: Chassis Power Control: Up/On 2024-03-26T06:08:22.158 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:22.166 DEBUG:teuthology.orchestra.console.smithi170:power on output: Chassis Power Control: Up/On 2024-03-26T06:08:22.166 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:22.174 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off 2024-03-26T06:08:22.182 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is off 2024-03-26T06:08:22.187 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2024-03-26T06:08:26.178 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:26.185 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:26.192 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:26.288 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off 2024-03-26T06:08:26.294 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is off 2024-03-26T06:08:26.301 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2024-03-26T06:08:30.293 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:30.301 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:30.308 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T06:08:30.403 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on 2024-03-26T06:08:30.409 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is on 2024-03-26T06:08:30.417 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2024-03-26T06:08:30.504 INFO:teuthology.orchestra.console.smithi038:Power on completed 2024-03-26T06:08:30.510 INFO:teuthology.orchestra.console.smithi121:Power on completed 2024-03-26T06:08:30.518 INFO:teuthology.orchestra.console.smithi170:Power on completed 2024-03-26T06:08:30.606 INFO:teuthology.provision.fog.smithi038:Waiting for deploy to finish 2024-03-26T06:08:30.610 INFO:teuthology.provision.fog.smithi121:Waiting for deploy to finish 2024-03-26T06:08:30.619 INFO:teuthology.provision.fog.smithi170:Waiting for deploy to finish 2024-03-26T06:11:01.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:01.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:01.518 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:04.574 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121 2024-03-26T06:11:04.575 WARNING:teuthology.provision.fog.smithi038:[Errno None] Unable to connect to port 22 on 172.21.15.38 2024-03-26T06:11:04.575 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-03-26T06:11:10.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:10.582 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:10.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:13.630 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121 2024-03-26T06:11:13.662 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-03-26T06:11:13.662 WARNING:teuthology.provision.fog.smithi038:[Errno None] Unable to connect to port 22 on 172.21.15.38 2024-03-26T06:11:19.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:19.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:19.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:22.690 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121 2024-03-26T06:11:22.722 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-03-26T06:11:22.723 WARNING:teuthology.provision.fog.smithi038:[Errno None] Unable to connect to port 22 on 172.21.15.38 2024-03-26T06:11:28.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:28.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:28.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:31.774 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121 2024-03-26T06:11:31.775 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-03-26T06:11:31.806 WARNING:teuthology.provision.fog.smithi038:[Errno None] Unable to connect to port 22 on 172.21.15.38 2024-03-26T06:11:37.779 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:37.780 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:37.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:40.830 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121 2024-03-26T06:11:40.831 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-03-26T06:11:40.862 WARNING:teuthology.provision.fog.smithi038:[Errno None] Unable to connect to port 22 on 172.21.15.38 2024-03-26T06:11:46.838 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:46.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:46.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:49.918 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121 2024-03-26T06:11:49.919 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-03-26T06:11:55.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:11:55.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:12:46.912 WARNING:teuthology.provision.fog.smithi038:timed out 2024-03-26T06:12:52.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:12:53.167 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-26T06:12:53.168 DEBUG:teuthology.orchestra.run.smithi038:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-26T06:12:53.494 INFO:teuthology.provision.fog.smithi038:Node is ready 2024-03-26T06:12:53.495 DEBUG:teuthology.orchestra.run.smithi038:> hostname 2024-03-26T06:12:53.510 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038.front.sepia.ceph.com 2024-03-26T06:12:53.510 DEBUG:teuthology.orchestra.run.smithi038:> grep smithi038.front.sepia.ceph.com /etc/hosts 2024-03-26T06:12:53.565 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38 smithi038.front.sepia.ceph.com smithi038 2024-03-26T06:12:53.566 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname smithi038 2024-03-26T06:12:53.675 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/smithi038.front.sepia.ceph.com/smithi038/g' /etc/hosts 2024-03-26T06:12:53.706 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/172.21.15.38/172.21.15.38/g' /etc/hosts 2024-03-26T06:12:53.778 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i -e 's/smithi038.front.sepia.ceph.com/smithi038/g' /etc/hostname 2024-03-26T06:12:53.848 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname smithi038 2024-03-26T06:12:53.918 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2024-03-26T06:12:53.973 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream" 2024-03-26T06:12:53.973 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="9" 2024-03-26T06:12:53.973 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos" 2024-03-26T06:12:53.973 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora" 2024-03-26T06:12:53.973 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="9" 2024-03-26T06:12:53.973 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el9" 2024-03-26T06:12:53.974 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T06:12:53.974 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2024-03-26T06:12:53.974 INFO:teuthology.orchestra.run.smithi038.stdout:LOGO="fedora-logo-icon" 2024-03-26T06:12:53.974 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T06:12:53.974 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/" 2024-03-26T06:12:53.974 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T06:12:53.974 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T06:12:53.974 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T06:12:53.975 INFO:teuthology.provision.fog.smithi038:Deploy complete! 2024-03-26T06:12:53.976 DEBUG:teuthology.parallel:result is None 2024-03-26T06:12:55.930 WARNING:teuthology.provision.fog.smithi121:timed out 2024-03-26T06:12:55.930 WARNING:teuthology.provision.fog.smithi170:timed out 2024-03-26T06:13:01.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:13:01.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:13:02.183 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-26T06:13:02.183 DEBUG:teuthology.orchestra.run.smithi121:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-26T06:13:02.212 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-26T06:13:02.212 DEBUG:teuthology.orchestra.run.smithi170:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-26T06:13:02.469 INFO:teuthology.provision.fog.smithi121:Node is ready 2024-03-26T06:13:02.469 DEBUG:teuthology.orchestra.run.smithi121:> hostname 2024-03-26T06:13:02.483 INFO:teuthology.orchestra.run.smithi121.stdout:smithi121.front.sepia.ceph.com 2024-03-26T06:13:02.484 DEBUG:teuthology.orchestra.run.smithi121:> grep smithi121.front.sepia.ceph.com /etc/hosts 2024-03-26T06:13:02.538 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.15.121 smithi121.front.sepia.ceph.com smithi121 2024-03-26T06:13:02.538 DEBUG:teuthology.orchestra.run.smithi121:> sudo hostname smithi121 2024-03-26T06:13:02.543 INFO:teuthology.provision.fog.smithi170:Node is ready 2024-03-26T06:13:02.543 DEBUG:teuthology.orchestra.run.smithi170:> hostname 2024-03-26T06:13:02.557 INFO:teuthology.orchestra.run.smithi170.stdout:smithi170.front.sepia.ceph.com 2024-03-26T06:13:02.558 DEBUG:teuthology.orchestra.run.smithi170:> grep smithi170.front.sepia.ceph.com /etc/hosts 2024-03-26T06:13:02.614 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.170 smithi170.front.sepia.ceph.com smithi170 2024-03-26T06:13:02.614 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname smithi170 2024-03-26T06:13:02.658 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i -e 's/smithi121.front.sepia.ceph.com/smithi121/g' /etc/hosts 2024-03-26T06:13:02.685 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i -e 's/172.21.15.121/172.21.15.121/g' /etc/hosts 2024-03-26T06:13:02.725 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i -e 's/smithi170.front.sepia.ceph.com/smithi170/g' /etc/hosts 2024-03-26T06:13:02.752 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i -e 's/smithi121.front.sepia.ceph.com/smithi121/g' /etc/hostname 2024-03-26T06:13:02.753 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i -e 's/172.21.15.170/172.21.15.170/g' /etc/hosts 2024-03-26T06:13:02.819 DEBUG:teuthology.orchestra.run.smithi121:> sudo hostname smithi121 2024-03-26T06:13:02.820 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i -e 's/smithi170.front.sepia.ceph.com/smithi170/g' /etc/hostname 2024-03-26T06:13:02.886 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2024-03-26T06:13:02.887 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname smithi170 2024-03-26T06:13:02.939 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream" 2024-03-26T06:13:02.940 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="9" 2024-03-26T06:13:02.940 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos" 2024-03-26T06:13:02.940 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora" 2024-03-26T06:13:02.940 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="9" 2024-03-26T06:13:02.940 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el9" 2024-03-26T06:13:02.940 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T06:13:02.940 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2024-03-26T06:13:02.940 INFO:teuthology.orchestra.run.smithi121.stdout:LOGO="fedora-logo-icon" 2024-03-26T06:13:02.940 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T06:13:02.940 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/" 2024-03-26T06:13:02.940 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T06:13:02.940 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T06:13:02.940 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T06:13:02.941 INFO:teuthology.provision.fog.smithi121:Deploy complete! 2024-03-26T06:13:02.941 DEBUG:teuthology.parallel:result is None 2024-03-26T06:13:02.953 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-03-26T06:13:03.007 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream" 2024-03-26T06:13:03.007 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="9" 2024-03-26T06:13:03.007 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos" 2024-03-26T06:13:03.008 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora" 2024-03-26T06:13:03.008 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="9" 2024-03-26T06:13:03.008 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el9" 2024-03-26T06:13:03.008 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T06:13:03.008 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2024-03-26T06:13:03.008 INFO:teuthology.orchestra.run.smithi170.stdout:LOGO="fedora-logo-icon" 2024-03-26T06:13:03.008 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T06:13:03.008 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/" 2024-03-26T06:13:03.008 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T06:13:03.008 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T06:13:03.008 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T06:13:03.008 INFO:teuthology.provision.fog.smithi170:Deploy complete! 2024-03-26T06:13:03.009 DEBUG:teuthology.parallel:result is None 2024-03-26T06:13:03.220 INFO:teuthology.lock.ops:Checking smithi038.front.sepia.ceph.com 2024-03-26T06:13:03.220 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-26T06:13:03.305 INFO:teuthology.lock.ops:Checking smithi121.front.sepia.ceph.com 2024-03-26T06:13:03.305 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-26T06:13:03.331 INFO:teuthology.lock.ops:Checking smithi170.front.sepia.ceph.com 2024-03-26T06:13:03.331 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-26T06:13:03.348 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: set os type and version on server 2024-03-26T06:13:03.349 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-03-26T06:13:03.349 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:13:03.453 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-03-26T06:13:03.523 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-03-26T06:13:03.523 DEBUG:teuthology.orchestra.run.smithi038:> uname -m 2024-03-26T06:13:03.580 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2024-03-26T06:13:03.580 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2024-03-26T06:13:03.637 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream" 2024-03-26T06:13:03.637 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="9" 2024-03-26T06:13:03.637 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos" 2024-03-26T06:13:03.637 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora" 2024-03-26T06:13:03.637 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="9" 2024-03-26T06:13:03.637 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el9" 2024-03-26T06:13:03.637 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T06:13:03.637 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2024-03-26T06:13:03.637 INFO:teuthology.orchestra.run.smithi038.stdout:LOGO="fedora-logo-icon" 2024-03-26T06:13:03.638 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T06:13:03.638 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/" 2024-03-26T06:13:03.638 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T06:13:03.638 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T06:13:03.638 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T06:13:03.638 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2024-03-26T06:13:03.664 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: set os type and version on server 2024-03-26T06:13:03.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-03-26T06:13:03.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:13:03.752 DEBUG:teuthology.orchestra.run.smithi121:> true 2024-03-26T06:13:03.816 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-03-26T06:13:03.816 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2024-03-26T06:13:03.871 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2024-03-26T06:13:03.872 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2024-03-26T06:13:03.925 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream" 2024-03-26T06:13:03.925 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="9" 2024-03-26T06:13:03.926 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos" 2024-03-26T06:13:03.926 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora" 2024-03-26T06:13:03.926 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="9" 2024-03-26T06:13:03.926 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el9" 2024-03-26T06:13:03.926 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T06:13:03.926 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2024-03-26T06:13:03.926 INFO:teuthology.orchestra.run.smithi121.stdout:LOGO="fedora-logo-icon" 2024-03-26T06:13:03.926 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T06:13:03.926 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/" 2024-03-26T06:13:03.927 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T06:13:03.927 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T06:13:03.927 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T06:13:03.927 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2024-03-26T06:13:03.948 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: set os type and version on server 2024-03-26T06:13:03.948 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-03-26T06:13:03.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T06:13:04.033 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-03-26T06:13:04.101 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-03-26T06:13:04.101 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2024-03-26T06:13:04.156 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2024-03-26T06:13:04.157 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-03-26T06:13:04.211 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream" 2024-03-26T06:13:04.211 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="9" 2024-03-26T06:13:04.211 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos" 2024-03-26T06:13:04.211 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora" 2024-03-26T06:13:04.211 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="9" 2024-03-26T06:13:04.211 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el9" 2024-03-26T06:13:04.211 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T06:13:04.211 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2024-03-26T06:13:04.211 INFO:teuthology.orchestra.run.smithi170.stdout:LOGO="fedora-logo-icon" 2024-03-26T06:13:04.211 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T06:13:04.211 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/" 2024-03-26T06:13:04.212 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T06:13:04.212 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T06:13:04.212 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T06:13:04.212 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-03-26T06:13:04.231 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:13:04.327 INFO:teuthology.dispatcher.supervisor:Running job 7622752 2024-03-26T06:13:04.327 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622752 --name rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/ffsb}} -- /home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622752/orig.config.yaml 2024-03-26T06:13:04.330 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2024-03-26_05:47:39-fs-rishabh-2024mar8-2-testing-default-smithi/7622752 2024-03-26T06:13:04.330 INFO:teuthology.dispatcher.supervisor:Job PID: 773466 2024-03-26T06:13:04.330 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-03-26T06:15:04.421 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:17:04.603 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:19:04.767 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:21:05.133 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:23:05.303 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:25:05.499 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:27:05.663 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:29:05.831 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:31:06.000 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:33:06.171 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:35:06.340 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:37:06.511 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:39:06.687 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:41:07.059 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:43:07.170 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:45:07.335 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:47:07.508 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:49:07.685 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:51:07.883 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:53:08.047 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:55:08.198 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T06:55:08.293 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1