2024-02-21T07:39:35.431 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-21T07:39:35.432 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T07:39:35.604 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fadfd3c8c10>, signals=[15]) 2024-02-21T07:39:35.604 INFO:teuthology.lock.ops:Start node 'smithi022.front.sepia.ceph.com' reimaging 2024-02-21T07:39:35.605 INFO:teuthology.lock.ops:Updating [smithi022.front.sepia.ceph.com]: reset os type and version on server 2024-02-21T07:39:35.605 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2024-02-21T07:39:35.630 INFO:teuthology.lock.ops:Start node 'smithi094.front.sepia.ceph.com' reimaging 2024-02-21T07:39:35.631 INFO:teuthology.lock.ops:Updating [smithi094.front.sepia.ceph.com]: reset os type and version on server 2024-02-21T07:39:35.631 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-02-21T07:39:35.667 INFO:teuthology.lock.ops:Start node 'smithi149.front.sepia.ceph.com' reimaging 2024-02-21T07:39:35.667 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: reset os type and version on server 2024-02-21T07:39:35.667 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-02-21T07:39:35.751 DEBUG:teuthology.provision.fog.smithi022:Requesting image smithi_centos_9.stream (ID 10) 2024-02-21T07:39:35.784 DEBUG:teuthology.provision.fog.smithi094:Requesting image smithi_centos_9.stream (ID 10) 2024-02-21T07:39:35.794 DEBUG:teuthology.provision.fog.smithi149:Requesting image smithi_centos_9.stream (ID 10) 2024-02-21T07:39:35.797 INFO:teuthology.provision.fog.smithi022:Scheduling deploy of centos 9.stream 2024-02-21T07:39:35.833 INFO:teuthology.provision.fog.smithi149:Scheduling deploy of centos 9.stream 2024-02-21T07:39:35.836 INFO:teuthology.provision.fog.smithi094:Scheduling deploy of centos 9.stream 2024-02-21T07:39:36.092 INFO:teuthology.orchestra.console.smithi022:Power off 2024-02-21T07:39:36.093 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-21T07:39:36.111 INFO:teuthology.orchestra.console.smithi094:Power off 2024-02-21T07:39:36.112 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-21T07:39:36.124 DEBUG:teuthology.orchestra.console.smithi022:power off output: Chassis Power Control: Down/Off 2024-02-21T07:39:36.124 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-21T07:39:36.135 DEBUG:teuthology.orchestra.console.smithi094:power off output: Chassis Power Control: Down/Off 2024-02-21T07:39:36.136 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-21T07:39:36.149 INFO:teuthology.orchestra.console.smithi149:Power off 2024-02-21T07:39:36.149 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-21T07:39:36.156 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is off 2024-02-21T07:39:36.157 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off 2024-02-21T07:39:36.171 DEBUG:teuthology.orchestra.console.smithi149:power off output: Chassis Power Control: Down/Off 2024-02-21T07:39:36.171 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-21T07:39:36.198 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2024-02-21T07:39:36.257 INFO:teuthology.orchestra.console.smithi094:Power off completed 2024-02-21T07:39:36.257 INFO:teuthology.orchestra.console.smithi022:Power off completed 2024-02-21T07:39:36.300 INFO:teuthology.orchestra.console.smithi149:Power off completed 2024-02-21T07:39:36.358 INFO:teuthology.orchestra.console.smithi094:Power on 2024-02-21T07:39:36.358 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-21T07:39:36.365 INFO:teuthology.orchestra.console.smithi022:Power on 2024-02-21T07:39:36.365 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-21T07:39:36.379 DEBUG:teuthology.orchestra.console.smithi094:power on output: Chassis Power Control: Up/On 2024-02-21T07:39:36.379 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-21T07:39:36.389 DEBUG:teuthology.orchestra.console.smithi022:power on output: Chassis Power Control: Up/On 2024-02-21T07:39:36.389 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-21T07:39:36.401 INFO:teuthology.orchestra.console.smithi149:Power on 2024-02-21T07:39:36.401 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-21T07:39:36.413 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is off 2024-02-21T07:39:36.415 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off 2024-02-21T07:39:36.426 DEBUG:teuthology.orchestra.console.smithi149:power on output: Chassis Power Control: Up/On 2024-02-21T07:39:36.427 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-21T07:39:36.453 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2024-02-21T07:39:40.416 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-21T07:39:40.424 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-21T07:39:40.454 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-21T07:39:40.526 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is off 2024-02-21T07:39:40.533 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off 2024-02-21T07:39:40.564 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2024-02-21T07:39:44.532 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-21T07:39:44.539 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-21T07:39:44.565 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-21T07:39:44.642 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2024-02-21T07:39:44.648 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2024-02-21T07:39:44.677 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-02-21T07:39:44.743 INFO:teuthology.orchestra.console.smithi094:Power on completed 2024-02-21T07:39:44.749 INFO:teuthology.orchestra.console.smithi022:Power on completed 2024-02-21T07:39:44.777 INFO:teuthology.orchestra.console.smithi149:Power on completed 2024-02-21T07:39:44.845 INFO:teuthology.provision.fog.smithi094:Waiting for deploy to finish 2024-02-21T07:39:44.849 INFO:teuthology.provision.fog.smithi022:Waiting for deploy to finish 2024-02-21T07:39:44.878 INFO:teuthology.provision.fog.smithi149:Waiting for deploy to finish 2024-02-21T07:42:16.385 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:42:16.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:42:19.452 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22 2024-02-21T07:42:19.452 WARNING:teuthology.provision.fog.smithi094:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-02-21T07:42:25.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:42:25.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:42:28.508 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22 2024-02-21T07:42:28.540 WARNING:teuthology.provision.fog.smithi094:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-02-21T07:42:34.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:42:34.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:42:37.564 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22 2024-02-21T07:42:37.596 WARNING:teuthology.provision.fog.smithi094:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-02-21T07:42:43.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:42:43.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:42:46.620 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22 2024-02-21T07:42:46.652 WARNING:teuthology.provision.fog.smithi094:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-02-21T07:42:52.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:42:52.653 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:42:55.708 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22 2024-02-21T07:42:55.708 WARNING:teuthology.provision.fog.smithi094:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-02-21T07:43:01.710 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:43:01.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:43:04.764 WARNING:teuthology.provision.fog.smithi094:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-02-21T07:43:10.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:43:13.820 WARNING:teuthology.provision.fog.smithi094:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-02-21T07:43:16.734 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:43:19.804 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-02-21T07:43:19.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:43:25.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:43:28.892 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-02-21T07:43:30.108 WARNING:teuthology.provision.fog.smithi094:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-02-21T07:43:34.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:43:36.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:43:37.980 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-02-21T07:43:39.164 WARNING:teuthology.provision.fog.smithi094:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-02-21T07:43:43.984 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:43:45.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:43:47.036 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-02-21T07:43:48.220 WARNING:teuthology.provision.fog.smithi094:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-02-21T07:43:53.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:43:54.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:43:56.092 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-02-21T07:43:56.627 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-21T07:43:56.628 DEBUG:teuthology.orchestra.run.smithi094:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-21T07:44:01.713 WARNING:teuthology.provision.fog.smithi022:timed out 2024-02-21T07:44:02.093 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:44:05.148 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-02-21T07:44:07.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:44:08.013 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-21T07:44:08.014 DEBUG:teuthology.orchestra.run.smithi022:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-21T07:44:08.345 INFO:teuthology.provision.fog.smithi022:Node is ready 2024-02-21T07:44:08.346 DEBUG:teuthology.orchestra.run.smithi022:> hostname 2024-02-21T07:44:08.361 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022.front.sepia.ceph.com 2024-02-21T07:44:08.361 DEBUG:teuthology.orchestra.run.smithi022:> grep smithi022.front.sepia.ceph.com /etc/hosts 2024-02-21T07:44:08.417 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022 2024-02-21T07:44:08.417 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname smithi022 2024-02-21T07:44:08.538 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/smithi022.front.sepia.ceph.com/smithi022/g' /etc/hosts 2024-02-21T07:44:08.570 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/172.21.15.22/172.21.15.22/g' /etc/hosts 2024-02-21T07:44:08.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/smithi022.front.sepia.ceph.com/smithi022/g' /etc/hostname 2024-02-21T07:44:08.713 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname smithi022 2024-02-21T07:44:08.784 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2024-02-21T07:44:08.840 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream" 2024-02-21T07:44:08.840 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9" 2024-02-21T07:44:08.840 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2024-02-21T07:44:08.840 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2024-02-21T07:44:08.840 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9" 2024-02-21T07:44:08.841 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9" 2024-02-21T07:44:08.841 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-21T07:44:08.841 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2024-02-21T07:44:08.841 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon" 2024-02-21T07:44:08.841 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-21T07:44:08.841 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/" 2024-02-21T07:44:08.842 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-21T07:44:08.842 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-21T07:44:08.842 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-21T07:44:08.843 INFO:teuthology.provision.fog.smithi022:Deploy complete! 2024-02-21T07:44:08.845 DEBUG:teuthology.parallel:result is None 2024-02-21T07:44:11.149 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:44:16.955 INFO:teuthology.provision.fog.smithi094:Node is ready 2024-02-21T07:44:16.956 DEBUG:teuthology.orchestra.run.smithi094:> hostname 2024-02-21T07:44:16.974 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094.front.sepia.ceph.com 2024-02-21T07:44:16.974 DEBUG:teuthology.orchestra.run.smithi094:> grep smithi094.front.sepia.ceph.com /etc/hosts 2024-02-21T07:44:17.032 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94 smithi094.front.sepia.ceph.com smithi094 2024-02-21T07:44:17.033 DEBUG:teuthology.orchestra.run.smithi094:> sudo hostname smithi094 2024-02-21T07:44:17.156 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i -e 's/smithi094.front.sepia.ceph.com/smithi094/g' /etc/hosts 2024-02-21T07:44:17.187 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i -e 's/172.21.15.94/172.21.15.94/g' /etc/hosts 2024-02-21T07:44:17.256 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i -e 's/smithi094.front.sepia.ceph.com/smithi094/g' /etc/hostname 2024-02-21T07:44:17.324 DEBUG:teuthology.orchestra.run.smithi094:> sudo hostname smithi094 2024-02-21T07:44:17.391 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-02-21T07:44:17.446 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-02-21T07:44:17.446 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-02-21T07:44:17.447 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-02-21T07:44:17.447 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-02-21T07:44:17.447 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-02-21T07:44:17.447 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-02-21T07:44:17.447 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-21T07:44:17.447 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-02-21T07:44:17.447 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-02-21T07:44:17.447 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-21T07:44:17.447 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-02-21T07:44:17.447 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-21T07:44:17.447 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-21T07:44:17.447 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-21T07:44:17.448 INFO:teuthology.provision.fog.smithi094:Deploy complete! 2024-02-21T07:44:17.448 DEBUG:teuthology.parallel:result is None 2024-02-21T07:44:29.504 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-02-21T07:44:35.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:44:38.588 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-02-21T07:44:44.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:44:47.680 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-02-21T07:44:53.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:44:54.015 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-21T07:44:54.015 DEBUG:teuthology.orchestra.run.smithi149:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-21T07:45:14.357 INFO:teuthology.provision.fog.smithi149:Node is ready 2024-02-21T07:45:14.357 DEBUG:teuthology.orchestra.run.smithi149:> hostname 2024-02-21T07:45:14.375 INFO:teuthology.orchestra.run.smithi149.stdout:smithi149.front.sepia.ceph.com 2024-02-21T07:45:14.376 DEBUG:teuthology.orchestra.run.smithi149:> grep smithi149.front.sepia.ceph.com /etc/hosts 2024-02-21T07:45:14.433 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.15.149 smithi149.front.sepia.ceph.com smithi149 2024-02-21T07:45:14.433 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname smithi149 2024-02-21T07:45:14.561 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/smithi149.front.sepia.ceph.com/smithi149/g' /etc/hosts 2024-02-21T07:45:14.592 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/172.21.15.149/172.21.15.149/g' /etc/hosts 2024-02-21T07:45:14.662 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/smithi149.front.sepia.ceph.com/smithi149/g' /etc/hostname 2024-02-21T07:45:14.731 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname smithi149 2024-02-21T07:45:14.799 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-02-21T07:45:14.855 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2024-02-21T07:45:14.855 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9" 2024-02-21T07:45:14.855 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2024-02-21T07:45:14.855 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2024-02-21T07:45:14.855 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9" 2024-02-21T07:45:14.855 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9" 2024-02-21T07:45:14.855 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-21T07:45:14.855 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2024-02-21T07:45:14.855 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon" 2024-02-21T07:45:14.855 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-21T07:45:14.855 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2024-02-21T07:45:14.855 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-21T07:45:14.856 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-21T07:45:14.856 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-21T07:45:14.856 INFO:teuthology.provision.fog.smithi149:Deploy complete! 2024-02-21T07:45:14.856 DEBUG:teuthology.parallel:result is None 2024-02-21T07:45:15.078 INFO:teuthology.lock.ops:Checking smithi022.front.sepia.ceph.com 2024-02-21T07:45:15.078 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-21T07:45:15.109 INFO:teuthology.lock.ops:Checking smithi094.front.sepia.ceph.com 2024-02-21T07:45:15.109 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-21T07:45:15.133 INFO:teuthology.lock.ops:Checking smithi149.front.sepia.ceph.com 2024-02-21T07:45:15.134 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-21T07:45:15.152 INFO:teuthology.lock.ops:Updating [smithi022.front.sepia.ceph.com]: set os type and version on server 2024-02-21T07:45:15.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-02-21T07:45:15.153 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:45:15.264 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-02-21T07:45:15.446 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-02-21T07:45:15.447 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2024-02-21T07:45:15.502 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2024-02-21T07:45:15.502 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2024-02-21T07:45:15.557 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream" 2024-02-21T07:45:15.557 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9" 2024-02-21T07:45:15.558 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2024-02-21T07:45:15.558 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2024-02-21T07:45:15.558 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9" 2024-02-21T07:45:15.558 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9" 2024-02-21T07:45:15.558 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-21T07:45:15.559 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2024-02-21T07:45:15.559 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon" 2024-02-21T07:45:15.559 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-21T07:45:15.559 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/" 2024-02-21T07:45:15.559 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-21T07:45:15.559 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-21T07:45:15.559 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-21T07:45:15.560 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2024-02-21T07:45:15.584 INFO:teuthology.lock.ops:Updating [smithi094.front.sepia.ceph.com]: set os type and version on server 2024-02-21T07:45:15.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-02-21T07:45:15.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:45:15.675 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-02-21T07:45:15.842 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-02-21T07:45:15.842 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-02-21T07:45:15.896 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-02-21T07:45:15.897 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-02-21T07:45:15.950 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-02-21T07:45:15.950 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-02-21T07:45:15.951 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-02-21T07:45:15.951 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-02-21T07:45:15.951 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-02-21T07:45:15.951 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-02-21T07:45:15.951 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-21T07:45:15.951 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-02-21T07:45:15.951 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-02-21T07:45:15.951 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-21T07:45:15.951 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-02-21T07:45:15.951 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-21T07:45:15.951 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-21T07:45:15.951 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-21T07:45:15.952 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-02-21T07:45:15.975 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: set os type and version on server 2024-02-21T07:45:15.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-02-21T07:45:15.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-21T07:45:16.067 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-02-21T07:45:16.136 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-02-21T07:45:16.136 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2024-02-21T07:45:16.192 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2024-02-21T07:45:16.193 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-02-21T07:45:16.247 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2024-02-21T07:45:16.247 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9" 2024-02-21T07:45:16.248 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2024-02-21T07:45:16.248 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2024-02-21T07:45:16.248 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9" 2024-02-21T07:45:16.248 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9" 2024-02-21T07:45:16.248 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-21T07:45:16.248 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2024-02-21T07:45:16.248 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon" 2024-02-21T07:45:16.248 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-21T07:45:16.248 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2024-02-21T07:45:16.248 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-21T07:45:16.248 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-21T07:45:16.248 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-21T07:45:16.249 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-02-21T07:45:16.267 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T07:45:16.370 INFO:teuthology.dispatcher.supervisor:Running job 7569323 2024-02-21T07:45:16.370 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-02-21_07:17:34-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7569323 --name vshankar-2024-02-21_07:17:34-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi --description fs:workload/{0-centos_9.stream begin/{0-install 1-ceph 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} -- /home/teuthworker/archive/vshankar-2024-02-21_07:17:34-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7569323/orig.config.yaml 2024-02-21T07:45:16.373 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-02-21_07:17:34-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7569323 2024-02-21T07:45:16.374 INFO:teuthology.dispatcher.supervisor:Job PID: 300269 2024-02-21T07:45:16.374 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-02-21T07:47:16.471 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T07:49:16.647 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T07:51:16.821 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T07:53:16.988 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T07:55:17.163 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T07:57:17.328 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T07:59:17.501 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T08:01:17.608 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T08:03:17.785 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T08:05:17.954 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T08:07:18.118 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T08:09:18.251 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T08:11:18.426 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T08:13:18.633 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T08:15:18.800 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T08:17:18.965 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T08:19:19.133 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T08:21:19.311 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T08:23:19.462 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T08:25:19.631 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T08:27:19.792 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-21T08:27:19.861 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1