2024-02-26T04:26:42.745 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55
2024-02-26T04:26:42.746 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:26:42.930 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f630a7d4f40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f630b835c10>, signals=[15])
2024-02-26T04:26:42.930 INFO:teuthology.lock.ops:Start node 'smithi006.front.sepia.ceph.com' reimaging
2024-02-26T04:26:42.930 INFO:teuthology.lock.ops:Updating [smithi006.front.sepia.ceph.com]: reset os type and version on server
2024-02-26T04:26:42.930 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server
2024-02-26T04:26:42.957 INFO:teuthology.lock.ops:Start node 'smithi090.front.sepia.ceph.com' reimaging
2024-02-26T04:26:42.957 INFO:teuthology.lock.ops:Updating [smithi090.front.sepia.ceph.com]: reset os type and version on server
2024-02-26T04:26:42.958 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server
2024-02-26T04:26:42.973 INFO:teuthology.lock.ops:Start node 'smithi159.front.sepia.ceph.com' reimaging
2024-02-26T04:26:42.974 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: reset os type and version on server
2024-02-26T04:26:42.974 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server
2024-02-26T04:26:43.103 DEBUG:teuthology.provision.fog.smithi090:Requesting image smithi_centos_9.stream (ID 10)
2024-02-26T04:26:43.137 DEBUG:teuthology.provision.fog.smithi006:Requesting image smithi_centos_9.stream (ID 10)
2024-02-26T04:26:43.148 DEBUG:teuthology.provision.fog.smithi159:Requesting image smithi_centos_9.stream (ID 10)
2024-02-26T04:26:43.153 INFO:teuthology.provision.fog.smithi090:Scheduling deploy of centos 9.stream
2024-02-26T04:26:43.196 INFO:teuthology.provision.fog.smithi006:Scheduling deploy of centos 9.stream
2024-02-26T04:26:43.204 INFO:teuthology.provision.fog.smithi159:Scheduling deploy of centos 9.stream
2024-02-26T04:26:43.696 INFO:teuthology.orchestra.console.smithi090:Power off
2024-02-26T04:26:43.696 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-02-26T04:26:43.718 DEBUG:teuthology.orchestra.console.smithi090:power off output: Chassis Power Control: Down/Off
2024-02-26T04:26:43.719 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:43.737 INFO:teuthology.orchestra.console.smithi006:Power off
2024-02-26T04:26:43.737 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-02-26T04:26:43.751 INFO:teuthology.orchestra.console.smithi159:Power off
2024-02-26T04:26:43.751 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-02-26T04:26:43.759 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on
2024-02-26T04:26:43.759 DEBUG:teuthology.orchestra.console.smithi006:power off output: Chassis Power Control: Down/Off
2024-02-26T04:26:43.759 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:43.776 DEBUG:teuthology.orchestra.console.smithi159:power off output: Chassis Power Control: Down/Off
2024-02-26T04:26:43.776 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:43.785 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on
2024-02-26T04:26:43.797 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on
2024-02-26T04:26:47.764 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:47.786 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:47.797 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:47.875 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on
2024-02-26T04:26:47.895 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on
2024-02-26T04:26:47.906 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on
2024-02-26T04:26:51.878 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:51.896 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:51.908 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:51.988 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on
2024-02-26T04:26:52.004 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on
2024-02-26T04:26:52.016 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off
2024-02-26T04:26:52.116 INFO:teuthology.orchestra.console.smithi159:Power off completed
2024-02-26T04:26:52.217 INFO:teuthology.orchestra.console.smithi159:Power on
2024-02-26T04:26:52.217 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-02-26T04:26:52.238 DEBUG:teuthology.orchestra.console.smithi159:power on output: Chassis Power Control: Up/On
2024-02-26T04:26:52.238 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:52.260 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off
2024-02-26T04:26:55.992 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:56.005 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:56.101 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off
2024-02-26T04:26:56.113 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is off
2024-02-26T04:26:56.202 INFO:teuthology.orchestra.console.smithi090:Power off completed
2024-02-26T04:26:56.214 INFO:teuthology.orchestra.console.smithi006:Power off completed
2024-02-26T04:26:56.261 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:56.304 INFO:teuthology.orchestra.console.smithi090:Power on
2024-02-26T04:26:56.304 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-02-26T04:26:56.314 INFO:teuthology.orchestra.console.smithi006:Power on
2024-02-26T04:26:56.315 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-02-26T04:26:56.326 DEBUG:teuthology.orchestra.console.smithi090:power on output: Chassis Power Control: Up/On
2024-02-26T04:26:56.326 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:56.335 DEBUG:teuthology.orchestra.console.smithi006:power on output: Chassis Power Control: Up/On
2024-02-26T04:26:56.335 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:56.347 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off
2024-02-26T04:26:56.355 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is off
2024-02-26T04:26:56.369 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off
2024-02-26T04:27:00.352 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:27:00.358 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:27:00.370 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:27:00.460 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on
2024-02-26T04:27:00.467 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on
2024-02-26T04:27:00.479 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on
2024-02-26T04:27:00.561 INFO:teuthology.orchestra.console.smithi090:Power on completed
2024-02-26T04:27:00.568 INFO:teuthology.orchestra.console.smithi006:Power on completed
2024-02-26T04:27:00.580 INFO:teuthology.orchestra.console.smithi159:Power on completed
2024-02-26T04:27:00.663 INFO:teuthology.provision.fog.smithi090:Waiting for deploy to finish
2024-02-26T04:27:00.669 INFO:teuthology.provision.fog.smithi006:Waiting for deploy to finish
2024-02-26T04:27:00.681 INFO:teuthology.provision.fog.smithi159:Waiting for deploy to finish
2024-02-26T04:30:19.172 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:30:37.628 WARNING:teuthology.provision.fog.smithi006:[Errno None] Unable to connect to port 22 on 172.21.15.6
2024-02-26T04:30:43.633 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:30:46.684 WARNING:teuthology.provision.fog.smithi006:[Errno None] Unable to connect to port 22 on 172.21.15.6
2024-02-26T04:30:52.686 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:30:55.740 WARNING:teuthology.provision.fog.smithi006:[Errno None] Unable to connect to port 22 on 172.21.15.6
2024-02-26T04:31:01.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:31:04.828 WARNING:teuthology.provision.fog.smithi006:[Errno None] Unable to connect to port 22 on 172.21.15.6
2024-02-26T04:31:10.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:32:10.833 WARNING:teuthology.provision.fog.smithi006:timed out
2024-02-26T04:32:16.841 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:32:17.062 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-02-26T04:32:17.063 DEBUG:teuthology.orchestra.run.smithi006:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-02-26T04:32:17.379 INFO:teuthology.provision.fog.smithi006:Node is ready
2024-02-26T04:32:17.379 DEBUG:teuthology.orchestra.run.smithi006:> hostname
2024-02-26T04:32:17.393 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006.front.sepia.ceph.com
2024-02-26T04:32:17.394 DEBUG:teuthology.orchestra.run.smithi006:> grep smithi006.front.sepia.ceph.com /etc/hosts
2024-02-26T04:32:17.449 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6 smithi006.front.sepia.ceph.com smithi006
2024-02-26T04:32:17.449 DEBUG:teuthology.orchestra.run.smithi006:> sudo hostname smithi006
2024-02-26T04:32:17.572 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i -e 's/smithi006.front.sepia.ceph.com/smithi006/g' /etc/hosts
2024-02-26T04:32:17.603 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i -e 's/172.21.15.6/172.21.15.6/g' /etc/hosts
2024-02-26T04:32:17.674 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i -e 's/smithi006.front.sepia.ceph.com/smithi006/g' /etc/hostname
2024-02-26T04:32:17.744 DEBUG:teuthology.orchestra.run.smithi006:> sudo hostname smithi006
2024-02-26T04:32:17.817 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release
2024-02-26T04:32:17.875 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="CentOS Stream"
2024-02-26T04:32:17.876 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="9"
2024-02-26T04:32:17.876 INFO:teuthology.orchestra.run.smithi006.stdout:ID="centos"
2024-02-26T04:32:17.876 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="rhel fedora"
2024-02-26T04:32:17.876 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="9"
2024-02-26T04:32:17.876 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el9"
2024-02-26T04:32:17.876 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-26T04:32:17.876 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31"
2024-02-26T04:32:17.876 INFO:teuthology.orchestra.run.smithi006.stdout:LOGO="fedora-logo-icon"
2024-02-26T04:32:17.876 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-26T04:32:17.876 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://centos.org/"
2024-02-26T04:32:17.877 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-26T04:32:17.877 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-26T04:32:17.877 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-26T04:32:17.878 INFO:teuthology.provision.fog.smithi006:Deploy complete!
2024-02-26T04:32:17.879 DEBUG:teuthology.parallel:result is None
2024-02-26T04:33:41.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:33:44.576 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2024-02-26T04:33:50.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:33:53.663 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2024-02-26T04:33:59.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:02.748 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2024-02-26T04:34:08.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:11.804 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2024-02-26T04:34:17.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:20.892 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2024-02-26T04:34:26.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:29.948 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2024-02-26T04:34:35.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:39.734 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:42.812 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2024-02-26T04:34:48.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:51.900 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2024-02-26T04:34:57.905 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:35:00.956 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2024-02-26T04:35:06.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:35:10.044 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2024-02-26T04:35:16.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:35:19.100 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2024-02-26T04:35:25.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:35:28.156 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2024-02-26T04:35:34.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:35:35.963 WARNING:teuthology.provision.fog.smithi159:timed out
2024-02-26T04:35:37.244 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2024-02-26T04:35:41.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:35:42.272 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-02-26T04:35:42.272 DEBUG:teuthology.orchestra.run.smithi159:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-02-26T04:35:42.614 INFO:teuthology.provision.fog.smithi159:Node is ready
2024-02-26T04:35:42.615 DEBUG:teuthology.orchestra.run.smithi159:> hostname
2024-02-26T04:35:42.629 INFO:teuthology.orchestra.run.smithi159.stdout:smithi159.front.sepia.ceph.com
2024-02-26T04:35:42.630 DEBUG:teuthology.orchestra.run.smithi159:> grep smithi159.front.sepia.ceph.com /etc/hosts
2024-02-26T04:35:42.685 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159 smithi159.front.sepia.ceph.com smithi159
2024-02-26T04:35:42.685 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159
2024-02-26T04:35:42.794 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi159.front.sepia.ceph.com/smithi159/g' /etc/hosts
2024-02-26T04:35:42.821 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/172.21.15.159/172.21.15.159/g' /etc/hosts
2024-02-26T04:35:42.889 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi159.front.sepia.ceph.com/smithi159/g' /etc/hostname
2024-02-26T04:35:42.955 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159
2024-02-26T04:35:43.022 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release
2024-02-26T04:35:43.076 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream"
2024-02-26T04:35:43.077 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9"
2024-02-26T04:35:43.077 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos"
2024-02-26T04:35:43.077 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora"
2024-02-26T04:35:43.077 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9"
2024-02-26T04:35:43.077 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9"
2024-02-26T04:35:43.077 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-26T04:35:43.077 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31"
2024-02-26T04:35:43.077 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon"
2024-02-26T04:35:43.077 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-26T04:35:43.077 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/"
2024-02-26T04:35:43.077 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-26T04:35:43.077 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-26T04:35:43.077 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-26T04:35:43.078 INFO:teuthology.provision.fog.smithi159:Deploy complete!
2024-02-26T04:35:43.078 DEBUG:teuthology.parallel:result is None
2024-02-26T04:35:43.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:35:46.300 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2024-02-26T04:35:52.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:35:55.388 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2024-02-26T04:36:01.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:37:01.441 WARNING:teuthology.provision.fog.smithi090:timed out
2024-02-26T04:37:07.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:37:07.757 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-02-26T04:37:07.757 DEBUG:teuthology.orchestra.run.smithi090:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-02-26T04:37:08.100 INFO:teuthology.provision.fog.smithi090:Node is ready
2024-02-26T04:37:08.101 DEBUG:teuthology.orchestra.run.smithi090:> hostname
2024-02-26T04:37:08.115 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090.front.sepia.ceph.com
2024-02-26T04:37:08.115 DEBUG:teuthology.orchestra.run.smithi090:> grep smithi090.front.sepia.ceph.com /etc/hosts
2024-02-26T04:37:08.171 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90 smithi090.front.sepia.ceph.com smithi090
2024-02-26T04:37:08.171 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname smithi090
2024-02-26T04:37:08.285 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/smithi090.front.sepia.ceph.com/smithi090/g' /etc/hosts
2024-02-26T04:37:08.312 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/172.21.15.90/172.21.15.90/g' /etc/hosts
2024-02-26T04:37:08.379 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/smithi090.front.sepia.ceph.com/smithi090/g' /etc/hostname
2024-02-26T04:37:08.446 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname smithi090
2024-02-26T04:37:08.512 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release
2024-02-26T04:37:08.567 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream"
2024-02-26T04:37:08.567 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9"
2024-02-26T04:37:08.567 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos"
2024-02-26T04:37:08.567 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora"
2024-02-26T04:37:08.567 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9"
2024-02-26T04:37:08.567 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9"
2024-02-26T04:37:08.567 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-26T04:37:08.567 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31"
2024-02-26T04:37:08.567 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon"
2024-02-26T04:37:08.567 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-26T04:37:08.568 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/"
2024-02-26T04:37:08.568 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-26T04:37:08.568 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-26T04:37:08.568 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-26T04:37:08.568 INFO:teuthology.provision.fog.smithi090:Deploy complete!
2024-02-26T04:37:08.569 DEBUG:teuthology.parallel:result is None
2024-02-26T04:37:08.781 INFO:teuthology.lock.ops:Checking smithi006.front.sepia.ceph.com
2024-02-26T04:37:08.781 INFO:teuthology.lock.ops:New key found. Updating...
2024-02-26T04:37:08.810 INFO:teuthology.lock.ops:Checking smithi090.front.sepia.ceph.com
2024-02-26T04:37:08.810 INFO:teuthology.lock.ops:New key found. Updating...
2024-02-26T04:37:08.826 INFO:teuthology.lock.ops:Checking smithi159.front.sepia.ceph.com
2024-02-26T04:37:08.826 INFO:teuthology.lock.ops:New key found. Updating...
2024-02-26T04:37:08.844 INFO:teuthology.lock.ops:Updating [smithi006.front.sepia.ceph.com]: set os type and version on server
2024-02-26T04:37:08.844 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com'
2024-02-26T04:37:08.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:37:08.943 DEBUG:teuthology.orchestra.run.smithi006:> true
2024-02-26T04:37:09.126 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com'
2024-02-26T04:37:09.126 DEBUG:teuthology.orchestra.run.smithi006:> uname -m
2024-02-26T04:37:09.181 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64
2024-02-26T04:37:09.181 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release
2024-02-26T04:37:09.236 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="CentOS Stream"
2024-02-26T04:37:09.236 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="9"
2024-02-26T04:37:09.236 INFO:teuthology.orchestra.run.smithi006.stdout:ID="centos"
2024-02-26T04:37:09.236 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="rhel fedora"
2024-02-26T04:37:09.236 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="9"
2024-02-26T04:37:09.236 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el9"
2024-02-26T04:37:09.236 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-26T04:37:09.237 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31"
2024-02-26T04:37:09.237 INFO:teuthology.orchestra.run.smithi006.stdout:LOGO="fedora-logo-icon"
2024-02-26T04:37:09.237 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-26T04:37:09.237 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://centos.org/"
2024-02-26T04:37:09.237 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-26T04:37:09.237 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-26T04:37:09.237 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-26T04:37:09.237 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server
2024-02-26T04:37:09.260 INFO:teuthology.lock.ops:Updating [smithi090.front.sepia.ceph.com]: set os type and version on server
2024-02-26T04:37:09.260 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com'
2024-02-26T04:37:09.260 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:37:09.342 DEBUG:teuthology.orchestra.run.smithi090:> true
2024-02-26T04:37:09.410 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com'
2024-02-26T04:37:09.410 DEBUG:teuthology.orchestra.run.smithi090:> uname -m
2024-02-26T04:37:09.465 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64
2024-02-26T04:37:09.465 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release
2024-02-26T04:37:09.519 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream"
2024-02-26T04:37:09.519 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9"
2024-02-26T04:37:09.519 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos"
2024-02-26T04:37:09.519 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora"
2024-02-26T04:37:09.519 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9"
2024-02-26T04:37:09.519 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9"
2024-02-26T04:37:09.519 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-26T04:37:09.519 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31"
2024-02-26T04:37:09.519 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon"
2024-02-26T04:37:09.519 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-26T04:37:09.519 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/"
2024-02-26T04:37:09.519 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-26T04:37:09.519 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-26T04:37:09.520 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-26T04:37:09.520 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server
2024-02-26T04:37:09.543 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: set os type and version on server
2024-02-26T04:37:09.543 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com'
2024-02-26T04:37:09.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:37:09.669 DEBUG:teuthology.orchestra.run.smithi159:> true
2024-02-26T04:37:09.834 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com'
2024-02-26T04:37:09.834 DEBUG:teuthology.orchestra.run.smithi159:> uname -m
2024-02-26T04:37:09.889 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64
2024-02-26T04:37:09.889 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release
2024-02-26T04:37:09.943 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream"
2024-02-26T04:37:09.944 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9"
2024-02-26T04:37:09.944 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos"
2024-02-26T04:37:09.944 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora"
2024-02-26T04:37:09.944 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9"
2024-02-26T04:37:09.944 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9"
2024-02-26T04:37:09.944 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-26T04:37:09.944 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31"
2024-02-26T04:37:09.944 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon"
2024-02-26T04:37:09.944 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-26T04:37:09.944 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/"
2024-02-26T04:37:09.944 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-26T04:37:09.945 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-26T04:37:09.945 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-26T04:37:09.945 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server
2024-02-26T04:37:09.968 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:37:10.059 INFO:teuthology.dispatcher.supervisor:Running job 7573370
2024-02-26T04:37:10.059 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573370 --name vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi --description fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} -- /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573370/orig.config.yaml
2024-02-26T04:37:10.062 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573370
2024-02-26T04:37:10.062 INFO:teuthology.dispatcher.supervisor:Job PID: 1029190
2024-02-26T04:37:10.062 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-02-26T04:39:10.163 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:41:10.341 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:43:10.522 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:45:10.693 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:47:10.873 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:49:11.045 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:51:11.217 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:53:11.401 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:55:11.559 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:57:11.729 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:59:12.060 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:01:12.239 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:03:12.409 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:05:12.573 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:07:12.733 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:09:12.901 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:11:13.077 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:13:13.241 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:15:13.371 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:17:13.488 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:19:13.664 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:21:13.850 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:23:14.036 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:25:14.286 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:27:14.453 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:29:14.580 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:31:14.747 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:33:14.920 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:35:15.059 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:37:15.204 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:37:15.271 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1