2024-04-14T09:03:06.965 INFO:root:teuthology version: 0.0.1.dev265+gd7d6e60 2024-04-14T09:03:06.965 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T09:03:07.142 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd348510c10>, signals=[15]) 2024-04-14T09:03:07.142 INFO:teuthology.lock.ops:Start node 'smithi007.front.sepia.ceph.com' reimaging 2024-04-14T09:03:07.143 INFO:teuthology.lock.ops:Updating [smithi007.front.sepia.ceph.com]: reset os type and version on server 2024-04-14T09:03:07.143 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-04-14T09:03:07.166 INFO:teuthology.lock.ops:Start node 'smithi033.front.sepia.ceph.com' reimaging 2024-04-14T09:03:07.166 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: reset os type and version on server 2024-04-14T09:03:07.167 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-04-14T09:03:07.191 INFO:teuthology.lock.ops:Start node 'smithi112.front.sepia.ceph.com' reimaging 2024-04-14T09:03:07.191 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: reset os type and version on server 2024-04-14T09:03:07.191 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-04-14T09:03:07.285 DEBUG:teuthology.provision.fog.smithi007:Requesting image smithi_centos_9.stream (ID 10) 2024-04-14T09:03:07.310 DEBUG:teuthology.provision.fog.smithi033:Requesting image smithi_centos_9.stream (ID 10) 2024-04-14T09:03:07.343 DEBUG:teuthology.provision.fog.smithi112:Requesting image smithi_centos_9.stream (ID 10) 2024-04-14T09:03:07.351 INFO:teuthology.provision.fog.smithi007:Scheduling deploy of centos 9.stream 2024-04-14T09:03:07.359 INFO:teuthology.provision.fog.smithi033:Scheduling deploy of centos 9.stream 2024-04-14T09:03:07.391 INFO:teuthology.provision.fog.smithi112:Scheduling deploy of centos 9.stream 2024-04-14T09:03:07.579 INFO:teuthology.orchestra.console.smithi033:Power off 2024-04-14T09:03:07.579 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-14T09:03:07.606 DEBUG:teuthology.orchestra.console.smithi033:power off output: Chassis Power Control: Down/Off 2024-04-14T09:03:07.607 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:07.622 INFO:teuthology.orchestra.console.smithi007:Power off 2024-04-14T09:03:07.622 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-14T09:03:07.636 INFO:teuthology.orchestra.console.smithi112:Power off 2024-04-14T09:03:07.636 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-14T09:03:07.644 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-04-14T09:03:07.644 DEBUG:teuthology.orchestra.console.smithi007:power off output: Chassis Power Control: Down/Off 2024-04-14T09:03:07.644 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:07.661 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off 2024-04-14T09:03:07.662 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:07.672 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-04-14T09:03:07.683 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-04-14T09:03:11.647 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:11.673 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:11.684 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:11.758 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-04-14T09:03:11.783 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-04-14T09:03:11.793 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-04-14T09:03:15.762 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:15.784 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:15.794 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:15.871 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-04-14T09:03:15.893 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2024-04-14T09:03:15.903 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-04-14T09:03:15.995 INFO:teuthology.orchestra.console.smithi007:Power off completed 2024-04-14T09:03:16.095 INFO:teuthology.orchestra.console.smithi007:Power on 2024-04-14T09:03:16.096 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-14T09:03:16.117 DEBUG:teuthology.orchestra.console.smithi007:power on output: Chassis Power Control: Up/On 2024-04-14T09:03:16.117 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:16.141 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2024-04-14T09:03:19.874 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:19.905 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:19.984 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off 2024-04-14T09:03:20.014 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-04-14T09:03:20.085 INFO:teuthology.orchestra.console.smithi033:Power off completed 2024-04-14T09:03:20.115 INFO:teuthology.orchestra.console.smithi112:Power off completed 2024-04-14T09:03:20.141 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:20.186 INFO:teuthology.orchestra.console.smithi033:Power on 2024-04-14T09:03:20.187 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-14T09:03:20.213 DEBUG:teuthology.orchestra.console.smithi033:power on output: Chassis Power Control: Up/On 2024-04-14T09:03:20.214 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:20.223 INFO:teuthology.orchestra.console.smithi112:Power on 2024-04-14T09:03:20.224 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-14T09:03:20.240 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off 2024-04-14T09:03:20.246 DEBUG:teuthology.orchestra.console.smithi112:power on output: Chassis Power Control: Up/On 2024-04-14T09:03:20.246 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:20.255 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2024-04-14T09:03:20.270 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-04-14T09:03:24.242 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:24.257 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:24.270 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T09:03:24.351 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-04-14T09:03:24.366 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-04-14T09:03:24.379 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-04-14T09:03:24.453 INFO:teuthology.orchestra.console.smithi033:Power on completed 2024-04-14T09:03:24.467 INFO:teuthology.orchestra.console.smithi007:Power on completed 2024-04-14T09:03:24.481 INFO:teuthology.orchestra.console.smithi112:Power on completed 2024-04-14T09:03:24.554 INFO:teuthology.provision.fog.smithi033:Waiting for deploy to finish 2024-04-14T09:03:24.568 INFO:teuthology.provision.fog.smithi007:Waiting for deploy to finish 2024-04-14T09:03:24.582 INFO:teuthology.provision.fog.smithi112:Waiting for deploy to finish 2024-04-14T09:05:40.432 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:05:43.487 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-04-14T09:05:49.492 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:05:52.542 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-04-14T09:05:55.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:05:58.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:05:58.590 WARNING:teuthology.provision.fog.smithi007:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-04-14T09:06:01.598 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-04-14T09:06:04.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:06:07.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:06:07.646 WARNING:teuthology.provision.fog.smithi007:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-04-14T09:06:10.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:06:10.654 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-04-14T09:06:13.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:06:13.694 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-04-14T09:06:16.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:06:16.707 WARNING:teuthology.provision.fog.smithi007:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-04-14T09:06:19.699 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:06:19.714 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-04-14T09:06:22.710 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:06:22.750 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-04-14T09:06:25.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:06:25.790 WARNING:teuthology.provision.fog.smithi007:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-04-14T09:06:28.754 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:06:28.802 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-04-14T09:06:31.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:06:31.806 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-04-14T09:06:34.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:06:34.846 WARNING:teuthology.provision.fog.smithi007:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-04-14T09:06:37.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:06:40.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:06:40.866 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-04-14T09:06:46.871 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:06:49.950 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-04-14T09:06:55.954 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:06:59.006 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-04-14T09:07:05.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:07:34.840 WARNING:teuthology.provision.fog.smithi033:timed out 2024-04-14T09:07:40.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:07:40.852 WARNING:teuthology.provision.fog.smithi007:timed out 2024-04-14T09:07:41.074 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-14T09:07:41.075 DEBUG:teuthology.orchestra.run.smithi033:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-14T09:07:41.438 INFO:teuthology.provision.fog.smithi033:Node is ready 2024-04-14T09:07:41.439 DEBUG:teuthology.orchestra.run.smithi033:> hostname 2024-04-14T09:07:41.454 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033.front.sepia.ceph.com 2024-04-14T09:07:41.454 DEBUG:teuthology.orchestra.run.smithi033:> grep smithi033.front.sepia.ceph.com /etc/hosts 2024-04-14T09:07:41.510 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33 smithi033.front.sepia.ceph.com smithi033 2024-04-14T09:07:41.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname smithi033 2024-04-14T09:07:41.617 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/smithi033.front.sepia.ceph.com/smithi033/g' /etc/hosts 2024-04-14T09:07:41.649 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/172.21.15.33/172.21.15.33/g' /etc/hosts 2024-04-14T09:07:41.722 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/smithi033.front.sepia.ceph.com/smithi033/g' /etc/hostname 2024-04-14T09:07:41.793 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname smithi033 2024-04-14T09:07:41.864 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-04-14T09:07:41.918 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2024-04-14T09:07:41.919 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9" 2024-04-14T09:07:41.919 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2024-04-14T09:07:41.919 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2024-04-14T09:07:41.919 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9" 2024-04-14T09:07:41.919 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9" 2024-04-14T09:07:41.919 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T09:07:41.919 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-04-14T09:07:41.919 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon" 2024-04-14T09:07:41.919 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T09:07:41.919 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2024-04-14T09:07:41.919 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T09:07:41.919 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T09:07:41.920 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T09:07:41.920 INFO:teuthology.provision.fog.smithi033:Deploy complete! 2024-04-14T09:07:41.922 DEBUG:teuthology.parallel:result is None 2024-04-14T09:07:46.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:07:47.105 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-14T09:07:47.105 DEBUG:teuthology.orchestra.run.smithi007:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-14T09:07:47.475 INFO:teuthology.provision.fog.smithi007:Node is ready 2024-04-14T09:07:47.475 DEBUG:teuthology.orchestra.run.smithi007:> hostname 2024-04-14T09:07:47.492 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007.front.sepia.ceph.com 2024-04-14T09:07:47.492 DEBUG:teuthology.orchestra.run.smithi007:> grep smithi007.front.sepia.ceph.com /etc/hosts 2024-04-14T09:07:47.548 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7 smithi007.front.sepia.ceph.com smithi007 2024-04-14T09:07:47.548 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname smithi007 2024-04-14T09:07:47.652 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i -e 's/smithi007.front.sepia.ceph.com/smithi007/g' /etc/hosts 2024-04-14T09:07:47.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i -e 's/172.21.15.7/172.21.15.7/g' /etc/hosts 2024-04-14T09:07:47.757 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i -e 's/smithi007.front.sepia.ceph.com/smithi007/g' /etc/hostname 2024-04-14T09:07:47.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname smithi007 2024-04-14T09:07:47.900 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-04-14T09:07:47.955 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-04-14T09:07:47.956 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-04-14T09:07:47.956 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-04-14T09:07:47.956 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-04-14T09:07:47.956 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-04-14T09:07:47.956 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-04-14T09:07:47.956 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T09:07:47.956 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-04-14T09:07:47.956 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-04-14T09:07:47.956 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T09:07:47.956 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-04-14T09:07:47.956 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T09:07:47.957 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T09:07:47.957 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T09:07:47.957 INFO:teuthology.provision.fog.smithi007:Deploy complete! 2024-04-14T09:07:47.958 DEBUG:teuthology.parallel:result is None 2024-04-14T09:08:05.026 WARNING:teuthology.provision.fog.smithi112:timed out 2024-04-14T09:08:11.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:08:11.252 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-14T09:08:11.252 DEBUG:teuthology.orchestra.run.smithi112:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-14T09:08:11.586 INFO:teuthology.provision.fog.smithi112:Node is ready 2024-04-14T09:08:11.586 DEBUG:teuthology.orchestra.run.smithi112:> hostname 2024-04-14T09:08:11.603 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112.front.sepia.ceph.com 2024-04-14T09:08:11.603 DEBUG:teuthology.orchestra.run.smithi112:> grep smithi112.front.sepia.ceph.com /etc/hosts 2024-04-14T09:08:11.658 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112 smithi112.front.sepia.ceph.com smithi112 2024-04-14T09:08:11.658 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname smithi112 2024-04-14T09:08:11.755 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/smithi112.front.sepia.ceph.com/smithi112/g' /etc/hosts 2024-04-14T09:08:11.783 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/172.21.15.112/172.21.15.112/g' /etc/hosts 2024-04-14T09:08:11.851 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/smithi112.front.sepia.ceph.com/smithi112/g' /etc/hostname 2024-04-14T09:08:11.919 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname smithi112 2024-04-14T09:08:11.985 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-04-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-04-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-04-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-04-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-04-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-04-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-04-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-04-14T09:08:12.039 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-04-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-04-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T09:08:12.040 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T09:08:12.040 INFO:teuthology.provision.fog.smithi112:Deploy complete! 2024-04-14T09:08:12.041 DEBUG:teuthology.parallel:result is None 2024-04-14T09:08:12.264 INFO:teuthology.lock.ops:Checking smithi007.front.sepia.ceph.com 2024-04-14T09:08:12.264 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-14T09:08:12.287 INFO:teuthology.lock.ops:Checking smithi033.front.sepia.ceph.com 2024-04-14T09:08:12.287 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-14T09:08:12.303 INFO:teuthology.lock.ops:Checking smithi112.front.sepia.ceph.com 2024-04-14T09:08:12.303 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-14T09:08:12.321 INFO:teuthology.lock.ops:Updating [smithi007.front.sepia.ceph.com]: set os type and version on server 2024-04-14T09:08:12.322 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-04-14T09:08:12.322 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:08:12.425 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-04-14T09:08:12.605 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-04-14T09:08:12.605 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-04-14T09:08:12.660 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-04-14T09:08:12.660 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-04-14T09:08:12.714 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-04-14T09:08:12.714 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-04-14T09:08:12.714 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-04-14T09:08:12.715 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-04-14T09:08:12.715 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-04-14T09:08:12.715 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-04-14T09:08:12.715 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T09:08:12.715 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-04-14T09:08:12.715 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-04-14T09:08:12.715 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T09:08:12.715 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-04-14T09:08:12.715 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T09:08:12.715 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T09:08:12.715 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T09:08:12.716 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-04-14T09:08:12.736 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: set os type and version on server 2024-04-14T09:08:12.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-14T09:08:12.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:08:12.837 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-04-14T09:08:13.020 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-14T09:08:13.021 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-04-14T09:08:13.075 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-04-14T09:08:13.076 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-04-14T09:08:13.130 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2024-04-14T09:08:13.130 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9" 2024-04-14T09:08:13.130 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2024-04-14T09:08:13.130 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2024-04-14T09:08:13.131 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9" 2024-04-14T09:08:13.131 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9" 2024-04-14T09:08:13.131 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T09:08:13.131 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-04-14T09:08:13.131 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon" 2024-04-14T09:08:13.131 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T09:08:13.131 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2024-04-14T09:08:13.131 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T09:08:13.131 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T09:08:13.131 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T09:08:13.132 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-04-14T09:08:13.162 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: set os type and version on server 2024-04-14T09:08:13.162 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-14T09:08:13.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T09:08:13.247 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-04-14T09:08:13.322 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-14T09:08:13.323 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-04-14T09:08:13.379 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-04-14T09:08:13.380 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-04-14T09:08:13.436 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-04-14T09:08:13.436 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-04-14T09:08:13.436 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-04-14T09:08:13.437 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-04-14T09:08:13.437 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-04-14T09:08:13.437 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-04-14T09:08:13.437 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T09:08:13.437 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-04-14T09:08:13.437 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-04-14T09:08:13.437 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T09:08:13.437 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-04-14T09:08:13.437 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T09:08:13.437 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T09:08:13.437 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T09:08:13.438 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-04-14T09:08:13.462 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T09:08:13.555 INFO:teuthology.dispatcher.supervisor:Running job 7638435 2024-04-14T09:08:13.555 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_d7d6e60867cadf19fbfd7655a551ae7d58060757/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638435 --name teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/test_o_trunc}} -- /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638435/orig.config.yaml 2024-04-14T09:08:13.558 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638435 2024-04-14T09:08:13.558 INFO:teuthology.dispatcher.supervisor:Job PID: 2347231 2024-04-14T09:08:13.558 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-14T09:10:13.659 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T09:12:13.835 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T09:14:14.021 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T09:16:14.193 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T09:18:14.368 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T09:20:14.547 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T09:22:14.723 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T09:22:14.800 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2024-04-14T09:22:14.957 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi007 but it was locked by another job: /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638468 2024-04-14T09:22:14.958 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi033 but it was locked by another job: /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638468 2024-04-14T09:22:14.958 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi112 but it was locked by another job: /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638474