2024-04-16T16:34:03.395 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-16T16:34:03.396 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T16:34:03.586 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb7cd86ec10>, signals=[15]) 2024-04-16T16:34:03.587 INFO:teuthology.lock.ops:Start node 'smithi042.front.sepia.ceph.com' reimaging 2024-04-16T16:34:03.587 INFO:teuthology.lock.ops:Updating [smithi042.front.sepia.ceph.com]: reset os type and version on server 2024-04-16T16:34:03.587 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-04-16T16:34:03.608 INFO:teuthology.lock.ops:Start node 'smithi079.front.sepia.ceph.com' reimaging 2024-04-16T16:34:03.608 INFO:teuthology.lock.ops:Updating [smithi079.front.sepia.ceph.com]: reset os type and version on server 2024-04-16T16:34:03.608 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2024-04-16T16:34:03.632 INFO:teuthology.lock.ops:Start node 'smithi183.front.sepia.ceph.com' reimaging 2024-04-16T16:34:03.632 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: reset os type and version on server 2024-04-16T16:34:03.632 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-04-16T16:34:03.713 DEBUG:teuthology.provision.fog.smithi042:Requesting image smithi_centos_9.stream (ID 10) 2024-04-16T16:34:03.741 DEBUG:teuthology.provision.fog.smithi079:Requesting image smithi_centos_9.stream (ID 10) 2024-04-16T16:34:03.753 DEBUG:teuthology.provision.fog.smithi183:Requesting image smithi_centos_9.stream (ID 10) 2024-04-16T16:34:03.771 INFO:teuthology.provision.fog.smithi042:Scheduling deploy of centos 9.stream 2024-04-16T16:34:03.782 INFO:teuthology.provision.fog.smithi079:Scheduling deploy of centos 9.stream 2024-04-16T16:34:03.799 INFO:teuthology.provision.fog.smithi183:Scheduling deploy of centos 9.stream 2024-04-16T16:34:04.034 INFO:teuthology.orchestra.console.smithi042:Power off 2024-04-16T16:34:04.034 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T16:34:04.048 INFO:teuthology.orchestra.console.smithi079:Power off 2024-04-16T16:34:04.049 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T16:34:04.061 INFO:teuthology.orchestra.console.smithi183:Power off 2024-04-16T16:34:04.061 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T16:34:04.069 DEBUG:teuthology.orchestra.console.smithi042:power off output: Chassis Power Control: Down/Off 2024-04-16T16:34:04.070 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T16:34:04.077 DEBUG:teuthology.orchestra.console.smithi079:power off output: Chassis Power Control: Down/Off 2024-04-16T16:34:04.077 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T16:34:04.085 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off 2024-04-16T16:34:04.086 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T16:34:04.093 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is off 2024-04-16T16:34:04.097 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2024-04-16T16:34:04.106 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2024-04-16T16:34:04.194 INFO:teuthology.orchestra.console.smithi042:Power off completed 2024-04-16T16:34:04.207 INFO:teuthology.orchestra.console.smithi183:Power off completed 2024-04-16T16:34:04.295 INFO:teuthology.orchestra.console.smithi042:Power on 2024-04-16T16:34:04.295 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-16T16:34:04.307 INFO:teuthology.orchestra.console.smithi183:Power on 2024-04-16T16:34:04.308 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-16T16:34:04.316 DEBUG:teuthology.orchestra.console.smithi042:power on output: Chassis Power Control: Up/On 2024-04-16T16:34:04.316 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T16:34:04.327 DEBUG:teuthology.orchestra.console.smithi183:power on output: Chassis Power Control: Up/On 2024-04-16T16:34:04.328 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T16:34:04.336 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is off 2024-04-16T16:34:04.348 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2024-04-16T16:34:08.102 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T16:34:08.210 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2024-04-16T16:34:08.337 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T16:34:08.349 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T16:34:08.446 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is off 2024-04-16T16:34:08.457 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2024-04-16T16:34:12.215 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T16:34:12.324 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off 2024-04-16T16:34:12.425 INFO:teuthology.orchestra.console.smithi079:Power off completed 2024-04-16T16:34:12.448 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T16:34:12.458 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T16:34:12.526 INFO:teuthology.orchestra.console.smithi079:Power on 2024-04-16T16:34:12.527 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-16T16:34:12.547 DEBUG:teuthology.orchestra.console.smithi079:power on output: Chassis Power Control: Up/On 2024-04-16T16:34:12.547 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T16:34:12.556 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is on 2024-04-16T16:34:12.566 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2024-04-16T16:34:12.568 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off 2024-04-16T16:34:12.657 INFO:teuthology.orchestra.console.smithi042:Power on completed 2024-04-16T16:34:12.667 INFO:teuthology.orchestra.console.smithi183:Power on completed 2024-04-16T16:34:12.758 INFO:teuthology.provision.fog.smithi042:Waiting for deploy to finish 2024-04-16T16:34:12.767 INFO:teuthology.provision.fog.smithi183:Waiting for deploy to finish 2024-04-16T16:34:16.573 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T16:34:16.682 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off 2024-04-16T16:34:20.687 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T16:34:20.795 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2024-04-16T16:34:20.896 INFO:teuthology.orchestra.console.smithi079:Power on completed 2024-04-16T16:34:20.996 INFO:teuthology.provision.fog.smithi079:Waiting for deploy to finish 2024-04-16T16:36:43.676 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:36:43.685 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:36:46.751 WARNING:teuthology.provision.fog.smithi042:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-04-16T16:36:46.751 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2024-04-16T16:36:51.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:36:52.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:36:52.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:36:55.806 WARNING:teuthology.provision.fog.smithi042:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-04-16T16:36:55.807 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2024-04-16T16:37:01.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:37:01.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:37:04.894 WARNING:teuthology.provision.fog.smithi042:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-04-16T16:37:04.895 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2024-04-16T16:37:10.302 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-04-16T16:37:10.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:37:10.899 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:37:13.950 WARNING:teuthology.provision.fog.smithi042:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-04-16T16:37:13.951 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2024-04-16T16:37:16.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:37:19.358 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-04-16T16:37:19.954 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:37:19.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:37:23.006 WARNING:teuthology.provision.fog.smithi042:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-04-16T16:37:23.007 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2024-04-16T16:37:25.362 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:37:28.414 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-04-16T16:37:29.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:37:29.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:37:32.062 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183 2024-04-16T16:37:34.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:37:37.470 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-04-16T16:37:38.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:37:43.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:37:46.558 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2024-04-16T16:37:52.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:38:29.047 WARNING:teuthology.provision.fog.smithi042:timed out 2024-04-16T16:38:35.053 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:38:35.288 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-16T16:38:35.288 DEBUG:teuthology.orchestra.run.smithi042:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-16T16:38:35.613 INFO:teuthology.provision.fog.smithi042:Node is ready 2024-04-16T16:38:35.614 DEBUG:teuthology.orchestra.run.smithi042:> hostname 2024-04-16T16:38:35.629 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042.front.sepia.ceph.com 2024-04-16T16:38:35.629 DEBUG:teuthology.orchestra.run.smithi042:> grep smithi042.front.sepia.ceph.com /etc/hosts 2024-04-16T16:38:35.684 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42 smithi042.front.sepia.ceph.com smithi042 2024-04-16T16:38:35.685 DEBUG:teuthology.orchestra.run.smithi042:> sudo hostname smithi042 2024-04-16T16:38:35.802 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i -e 's/smithi042.front.sepia.ceph.com/smithi042/g' /etc/hosts 2024-04-16T16:38:35.834 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i -e 's/172.21.15.42/172.21.15.42/g' /etc/hosts 2024-04-16T16:38:35.905 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i -e 's/smithi042.front.sepia.ceph.com/smithi042/g' /etc/hostname 2024-04-16T16:38:35.976 DEBUG:teuthology.orchestra.run.smithi042:> sudo hostname smithi042 2024-04-16T16:38:36.047 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-04-16T16:38:36.103 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="CentOS Stream" 2024-04-16T16:38:36.103 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="9" 2024-04-16T16:38:36.103 INFO:teuthology.orchestra.run.smithi042.stdout:ID="centos" 2024-04-16T16:38:36.103 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="rhel fedora" 2024-04-16T16:38:36.103 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="9" 2024-04-16T16:38:36.103 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el9" 2024-04-16T16:38:36.103 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T16:38:36.103 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2024-04-16T16:38:36.103 INFO:teuthology.orchestra.run.smithi042.stdout:LOGO="fedora-logo-icon" 2024-04-16T16:38:36.103 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T16:38:36.103 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://centos.org/" 2024-04-16T16:38:36.104 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T16:38:36.104 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T16:38:36.104 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T16:38:36.104 INFO:teuthology.provision.fog.smithi042:Deploy complete! 2024-04-16T16:38:36.106 DEBUG:teuthology.parallel:result is None 2024-04-16T16:38:38.069 WARNING:teuthology.provision.fog.smithi183:timed out 2024-04-16T16:38:44.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:38:44.313 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-16T16:38:44.313 DEBUG:teuthology.orchestra.run.smithi183:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-16T16:38:44.612 INFO:teuthology.provision.fog.smithi183:Node is ready 2024-04-16T16:38:44.612 DEBUG:teuthology.orchestra.run.smithi183:> hostname 2024-04-16T16:38:44.626 INFO:teuthology.orchestra.run.smithi183.stdout:smithi183.front.sepia.ceph.com 2024-04-16T16:38:44.627 DEBUG:teuthology.orchestra.run.smithi183:> grep smithi183.front.sepia.ceph.com /etc/hosts 2024-04-16T16:38:44.681 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.15.183 smithi183.front.sepia.ceph.com smithi183 2024-04-16T16:38:44.682 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname smithi183 2024-04-16T16:38:44.805 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/smithi183.front.sepia.ceph.com/smithi183/g' /etc/hosts 2024-04-16T16:38:44.832 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/172.21.15.183/172.21.15.183/g' /etc/hosts 2024-04-16T16:38:44.899 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/smithi183.front.sepia.ceph.com/smithi183/g' /etc/hostname 2024-04-16T16:38:44.966 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname smithi183 2024-04-16T16:38:45.032 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-04-16T16:38:45.086 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-04-16T16:38:45.086 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-04-16T16:38:45.086 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-04-16T16:38:45.086 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-04-16T16:38:45.086 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-04-16T16:38:45.087 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-04-16T16:38:45.087 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T16:38:45.087 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-04-16T16:38:45.087 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-04-16T16:38:45.087 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T16:38:45.087 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-04-16T16:38:45.087 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T16:38:45.087 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T16:38:45.087 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T16:38:45.087 INFO:teuthology.provision.fog.smithi183:Deploy complete! 2024-04-16T16:38:45.088 DEBUG:teuthology.parallel:result is None 2024-04-16T16:38:52.575 WARNING:teuthology.provision.fog.smithi079:timed out 2024-04-16T16:38:58.582 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:38:58.787 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-16T16:38:58.787 DEBUG:teuthology.orchestra.run.smithi079:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-16T16:38:59.087 INFO:teuthology.provision.fog.smithi079:Node is ready 2024-04-16T16:38:59.087 DEBUG:teuthology.orchestra.run.smithi079:> hostname 2024-04-16T16:38:59.101 INFO:teuthology.orchestra.run.smithi079.stdout:smithi079.front.sepia.ceph.com 2024-04-16T16:38:59.101 DEBUG:teuthology.orchestra.run.smithi079:> grep smithi079.front.sepia.ceph.com /etc/hosts 2024-04-16T16:38:59.156 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79 smithi079.front.sepia.ceph.com smithi079 2024-04-16T16:38:59.156 DEBUG:teuthology.orchestra.run.smithi079:> sudo hostname smithi079 2024-04-16T16:38:59.265 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/smithi079.front.sepia.ceph.com/smithi079/g' /etc/hosts 2024-04-16T16:38:59.292 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/172.21.15.79/172.21.15.79/g' /etc/hosts 2024-04-16T16:38:59.359 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/smithi079.front.sepia.ceph.com/smithi079/g' /etc/hostname 2024-04-16T16:38:59.426 DEBUG:teuthology.orchestra.run.smithi079:> sudo hostname smithi079 2024-04-16T16:38:59.492 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2024-04-16T16:38:59.545 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Stream" 2024-04-16T16:38:59.546 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="9" 2024-04-16T16:38:59.546 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos" 2024-04-16T16:38:59.546 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora" 2024-04-16T16:38:59.546 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="9" 2024-04-16T16:38:59.546 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el9" 2024-04-16T16:38:59.546 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T16:38:59.546 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2024-04-16T16:38:59.546 INFO:teuthology.orchestra.run.smithi079.stdout:LOGO="fedora-logo-icon" 2024-04-16T16:38:59.546 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T16:38:59.546 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://centos.org/" 2024-04-16T16:38:59.546 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T16:38:59.546 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T16:38:59.547 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T16:38:59.547 INFO:teuthology.provision.fog.smithi079:Deploy complete! 2024-04-16T16:38:59.547 DEBUG:teuthology.parallel:result is None 2024-04-16T16:38:59.753 INFO:teuthology.lock.ops:Checking smithi042.front.sepia.ceph.com 2024-04-16T16:38:59.754 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-16T16:38:59.778 INFO:teuthology.lock.ops:Checking smithi079.front.sepia.ceph.com 2024-04-16T16:38:59.778 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-16T16:38:59.803 INFO:teuthology.lock.ops:Checking smithi183.front.sepia.ceph.com 2024-04-16T16:38:59.803 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-16T16:38:59.821 INFO:teuthology.lock.ops:Updating [smithi042.front.sepia.ceph.com]: set os type and version on server 2024-04-16T16:38:59.821 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-16T16:38:59.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:38:59.923 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-04-16T16:39:00.108 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-16T16:39:00.109 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2024-04-16T16:39:00.163 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2024-04-16T16:39:00.164 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-04-16T16:39:00.219 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="CentOS Stream" 2024-04-16T16:39:00.219 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="9" 2024-04-16T16:39:00.219 INFO:teuthology.orchestra.run.smithi042.stdout:ID="centos" 2024-04-16T16:39:00.219 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="rhel fedora" 2024-04-16T16:39:00.219 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="9" 2024-04-16T16:39:00.219 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el9" 2024-04-16T16:39:00.219 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T16:39:00.219 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2024-04-16T16:39:00.219 INFO:teuthology.orchestra.run.smithi042.stdout:LOGO="fedora-logo-icon" 2024-04-16T16:39:00.219 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T16:39:00.219 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://centos.org/" 2024-04-16T16:39:00.219 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T16:39:00.219 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T16:39:00.220 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T16:39:00.220 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-04-16T16:39:00.331 INFO:teuthology.lock.ops:Updating [smithi079.front.sepia.ceph.com]: set os type and version on server 2024-04-16T16:39:00.332 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-04-16T16:39:00.332 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:39:00.418 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-04-16T16:39:00.484 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-04-16T16:39:00.484 DEBUG:teuthology.orchestra.run.smithi079:> uname -m 2024-04-16T16:39:00.539 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2024-04-16T16:39:00.540 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2024-04-16T16:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Stream" 2024-04-16T16:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="9" 2024-04-16T16:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos" 2024-04-16T16:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora" 2024-04-16T16:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="9" 2024-04-16T16:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el9" 2024-04-16T16:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T16:39:00.594 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2024-04-16T16:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout:LOGO="fedora-logo-icon" 2024-04-16T16:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T16:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://centos.org/" 2024-04-16T16:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T16:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T16:39:00.595 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T16:39:00.595 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2024-04-16T16:39:00.620 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: set os type and version on server 2024-04-16T16:39:00.620 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-04-16T16:39:00.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:39:00.706 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-04-16T16:39:00.876 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-04-16T16:39:00.876 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-04-16T16:39:00.930 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-04-16T16:39:00.931 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-04-16T16:39:00.985 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-04-16T16:39:00.985 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-04-16T16:39:00.985 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-04-16T16:39:00.986 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-04-16T16:39:00.986 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-04-16T16:39:00.986 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-04-16T16:39:00.986 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T16:39:00.986 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-04-16T16:39:00.986 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-04-16T16:39:00.986 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T16:39:00.986 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-04-16T16:39:00.986 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T16:39:00.986 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T16:39:00.986 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T16:39:00.987 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-04-16T16:39:01.012 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T16:39:01.107 INFO:teuthology.dispatcher.supervisor:Running job 7658441 2024-04-16T16:39:01.107 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/bin/teuthology -v --owner scheduled_mchangir@teuthology --archive /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658441 --name mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} -- /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658441/orig.config.yaml 2024-04-16T16:39:01.110 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658441 2024-04-16T16:39:01.111 INFO:teuthology.dispatcher.supervisor:Job PID: 852654 2024-04-16T16:39:01.111 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-16T16:41:01.212 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T16:43:01.360 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T16:45:01.531 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T16:47:01.695 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T16:49:01.879 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T16:51:02.049 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T16:53:02.156 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T16:55:02.304 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T16:57:02.449 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T16:59:02.613 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T17:01:02.779 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T17:03:02.971 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T17:05:03.132 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T17:07:03.308 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T17:09:03.473 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T17:11:03.656 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T17:13:03.894 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T17:15:04.103 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T17:17:04.277 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T17:19:04.535 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T17:21:04.709 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T17:23:04.870 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T17:23:04.939 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1