2024-03-11T02:14:14.782 INFO:root:teuthology version: 0.0.1.dev262+ge691533
2024-03-11T02:14:14.783 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:14:14.959 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7ff817bdcf40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7ff818b3ec10>, signals=[15])
2024-03-11T02:14:14.959 INFO:teuthology.lock.ops:Start node 'smithi002.front.sepia.ceph.com' reimaging
2024-03-11T02:14:14.959 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: reset os type and version on server
2024-03-11T02:14:14.959 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server
2024-03-11T02:14:14.985 INFO:teuthology.lock.ops:Start node 'smithi033.front.sepia.ceph.com' reimaging
2024-03-11T02:14:14.986 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: reset os type and version on server
2024-03-11T02:14:14.986 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server
2024-03-11T02:14:15.010 INFO:teuthology.lock.ops:Start node 'smithi052.front.sepia.ceph.com' reimaging
2024-03-11T02:14:15.010 INFO:teuthology.lock.ops:Updating [smithi052.front.sepia.ceph.com]: reset os type and version on server
2024-03-11T02:14:15.010 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server
2024-03-11T02:14:15.113 DEBUG:teuthology.provision.fog.smithi002:Requesting image smithi_centos_9.stream (ID 10)
2024-03-11T02:14:15.117 DEBUG:teuthology.provision.fog.smithi033:Requesting image smithi_centos_9.stream (ID 10)
2024-03-11T02:14:15.137 DEBUG:teuthology.provision.fog.smithi052:Requesting image smithi_centos_9.stream (ID 10)
2024-03-11T02:14:15.160 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of centos 9.stream
2024-03-11T02:14:15.164 INFO:teuthology.provision.fog.smithi033:Scheduling deploy of centos 9.stream
2024-03-11T02:14:15.183 INFO:teuthology.provision.fog.smithi052:Scheduling deploy of centos 9.stream
2024-03-11T02:14:15.459 INFO:teuthology.orchestra.console.smithi002:Power off
2024-03-11T02:14:15.459 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-11T02:14:15.474 INFO:teuthology.orchestra.console.smithi052:Power off
2024-03-11T02:14:15.474 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-11T02:14:15.488 INFO:teuthology.orchestra.console.smithi033:Power off
2024-03-11T02:14:15.488 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-11T02:14:15.497 DEBUG:teuthology.orchestra.console.smithi002:power off output: Chassis Power Control: Down/Off
2024-03-11T02:14:15.497 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:15.504 DEBUG:teuthology.orchestra.console.smithi052:power off output: Chassis Power Control: Down/Off
2024-03-11T02:14:15.505 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:15.513 DEBUG:teuthology.orchestra.console.smithi033:power off output: Chassis Power Control: Down/Off
2024-03-11T02:14:15.513 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:15.522 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on
2024-03-11T02:14:15.527 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on
2024-03-11T02:14:15.538 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on
2024-03-11T02:14:19.526 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:19.534 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:19.542 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:19.637 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on
2024-03-11T02:14:19.646 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on
2024-03-11T02:14:19.653 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on
2024-03-11T02:14:23.639 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:23.647 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:23.654 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:23.749 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off
2024-03-11T02:14:23.756 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is off
2024-03-11T02:14:23.762 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on
2024-03-11T02:14:23.850 INFO:teuthology.orchestra.console.smithi002:Power off completed
2024-03-11T02:14:23.858 INFO:teuthology.orchestra.console.smithi052:Power off completed
2024-03-11T02:14:23.952 INFO:teuthology.orchestra.console.smithi002:Power on
2024-03-11T02:14:23.952 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-11T02:14:23.965 INFO:teuthology.orchestra.console.smithi052:Power on
2024-03-11T02:14:23.965 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-11T02:14:23.981 DEBUG:teuthology.orchestra.console.smithi002:power on output: Chassis Power Control: Up/On
2024-03-11T02:14:23.982 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:23.992 DEBUG:teuthology.orchestra.console.smithi052:power on output: Chassis Power Control: Up/On
2024-03-11T02:14:23.992 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:24.007 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off
2024-03-11T02:14:24.019 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is off
2024-03-11T02:14:27.766 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:27.875 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off
2024-03-11T02:14:27.976 INFO:teuthology.orchestra.console.smithi033:Power off completed
2024-03-11T02:14:28.008 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:28.019 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:28.077 INFO:teuthology.orchestra.console.smithi033:Power on
2024-03-11T02:14:28.077 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-11T02:14:28.101 DEBUG:teuthology.orchestra.console.smithi033:power on output: Chassis Power Control: Up/On
2024-03-11T02:14:28.102 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:28.119 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off
2024-03-11T02:14:28.127 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is off
2024-03-11T02:14:28.128 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off
2024-03-11T02:14:32.122 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:32.129 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:32.135 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T02:14:32.232 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on
2024-03-11T02:14:32.237 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on
2024-03-11T02:14:32.245 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on
2024-03-11T02:14:32.333 INFO:teuthology.orchestra.console.smithi002:Power on completed
2024-03-11T02:14:32.337 INFO:teuthology.orchestra.console.smithi052:Power on completed
2024-03-11T02:14:32.346 INFO:teuthology.orchestra.console.smithi033:Power on completed
2024-03-11T02:14:32.434 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish
2024-03-11T02:14:32.438 INFO:teuthology.provision.fog.smithi052:Waiting for deploy to finish
2024-03-11T02:14:32.447 INFO:teuthology.provision.fog.smithi033:Waiting for deploy to finish
2024-03-11T02:16:48.492 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:16:48.626 ERROR:teuthology.orchestra.connection:Error authenticating with smithi033.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-11T02:17:03.549 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:17:03.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:17:06.626 WARNING:teuthology.provision.fog.smithi052:[Errno None] Unable to connect to port 22 on 172.21.15.52
2024-03-11T02:17:06.627 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2
2024-03-11T02:17:12.628 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:17:12.630 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:17:15.682 WARNING:teuthology.provision.fog.smithi052:[Errno None] Unable to connect to port 22 on 172.21.15.52
2024-03-11T02:17:15.714 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2
2024-03-11T02:17:21.684 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:17:21.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:17:24.734 WARNING:teuthology.provision.fog.smithi052:[Errno None] Unable to connect to port 22 on 172.21.15.52
2024-03-11T02:17:24.766 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2
2024-03-11T02:17:27.074 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33
2024-03-11T02:17:30.736 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:17:30.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:17:33.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:17:33.790 WARNING:teuthology.provision.fog.smithi052:[Errno None] Unable to connect to port 22 on 172.21.15.52
2024-03-11T02:17:33.822 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2
2024-03-11T02:17:36.126 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33
2024-03-11T02:17:39.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:17:39.824 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:17:42.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:17:42.850 WARNING:teuthology.provision.fog.smithi052:[Errno None] Unable to connect to port 22 on 172.21.15.52
2024-03-11T02:17:42.882 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2
2024-03-11T02:17:48.852 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:17:48.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:18:42.130 WARNING:teuthology.provision.fog.smithi033:timed out
2024-03-11T02:18:48.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:18:48.392 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-11T02:18:48.393 DEBUG:teuthology.orchestra.run.smithi033:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-11T02:18:48.739 INFO:teuthology.provision.fog.smithi033:Node is ready
2024-03-11T02:18:48.739 DEBUG:teuthology.orchestra.run.smithi033:> hostname
2024-03-11T02:18:48.754 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033.front.sepia.ceph.com
2024-03-11T02:18:48.754 DEBUG:teuthology.orchestra.run.smithi033:> grep smithi033.front.sepia.ceph.com /etc/hosts
2024-03-11T02:18:48.810 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33 smithi033.front.sepia.ceph.com smithi033
2024-03-11T02:18:48.810 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname smithi033
2024-03-11T02:18:48.854 WARNING:teuthology.provision.fog.smithi052:timed out
2024-03-11T02:18:48.886 WARNING:teuthology.provision.fog.smithi002:timed out
2024-03-11T02:18:48.918 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/smithi033.front.sepia.ceph.com/smithi033/g' /etc/hosts
2024-03-11T02:18:48.949 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/172.21.15.33/172.21.15.33/g' /etc/hosts
2024-03-11T02:18:49.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/smithi033.front.sepia.ceph.com/smithi033/g' /etc/hostname
2024-03-11T02:18:49.091 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname smithi033
2024-03-11T02:18:49.162 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release
2024-03-11T02:18:49.218 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream"
2024-03-11T02:18:49.219 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9"
2024-03-11T02:18:49.219 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos"
2024-03-11T02:18:49.219 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora"
2024-03-11T02:18:49.219 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9"
2024-03-11T02:18:49.219 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9"
2024-03-11T02:18:49.219 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-11T02:18:49.219 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31"
2024-03-11T02:18:49.219 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon"
2024-03-11T02:18:49.219 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-11T02:18:49.219 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/"
2024-03-11T02:18:49.220 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-11T02:18:49.220 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-11T02:18:49.220 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-11T02:18:49.221 INFO:teuthology.provision.fog.smithi033:Deploy complete!
2024-03-11T02:18:49.222 DEBUG:teuthology.parallel:result is None
2024-03-11T02:18:54.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:18:54.888 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:18:55.152 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-11T02:18:55.152 DEBUG:teuthology.orchestra.run.smithi002:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-11T02:18:55.156 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-11T02:18:55.156 DEBUG:teuthology.orchestra.run.smithi052:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-11T02:18:55.470 INFO:teuthology.provision.fog.smithi052:Node is ready
2024-03-11T02:18:55.470 DEBUG:teuthology.orchestra.run.smithi052:> hostname
2024-03-11T02:18:55.485 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052.front.sepia.ceph.com
2024-03-11T02:18:55.486 DEBUG:teuthology.orchestra.run.smithi052:> grep smithi052.front.sepia.ceph.com /etc/hosts
2024-03-11T02:18:55.497 INFO:teuthology.provision.fog.smithi002:Node is ready
2024-03-11T02:18:55.497 DEBUG:teuthology.orchestra.run.smithi002:> hostname
2024-03-11T02:18:55.512 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002.front.sepia.ceph.com
2024-03-11T02:18:55.512 DEBUG:teuthology.orchestra.run.smithi002:> grep smithi002.front.sepia.ceph.com /etc/hosts
2024-03-11T02:18:55.541 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52 smithi052.front.sepia.ceph.com smithi052
2024-03-11T02:18:55.542 DEBUG:teuthology.orchestra.run.smithi052:> sudo hostname smithi052
2024-03-11T02:18:55.570 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2 smithi002.front.sepia.ceph.com smithi002
2024-03-11T02:18:55.570 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname smithi002
2024-03-11T02:18:55.644 DEBUG:teuthology.orchestra.run.smithi052:> sudo sed -i -e 's/smithi052.front.sepia.ceph.com/smithi052/g' /etc/hosts
2024-03-11T02:18:55.675 DEBUG:teuthology.orchestra.run.smithi052:> sudo sed -i -e 's/172.21.15.52/172.21.15.52/g' /etc/hosts
2024-03-11T02:18:55.682 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/smithi002.front.sepia.ceph.com/smithi002/g' /etc/hosts
2024-03-11T02:18:55.714 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/172.21.15.2/172.21.15.2/g' /etc/hosts
2024-03-11T02:18:55.746 DEBUG:teuthology.orchestra.run.smithi052:> sudo sed -i -e 's/smithi052.front.sepia.ceph.com/smithi052/g' /etc/hostname
2024-03-11T02:18:55.784 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/smithi002.front.sepia.ceph.com/smithi002/g' /etc/hostname
2024-03-11T02:18:55.816 DEBUG:teuthology.orchestra.run.smithi052:> sudo hostname smithi052
2024-03-11T02:18:55.855 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname smithi002
2024-03-11T02:18:55.886 DEBUG:teuthology.orchestra.run.smithi052:> cat /etc/os-release
2024-03-11T02:18:55.925 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release
2024-03-11T02:18:55.941 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="CentOS Stream"
2024-03-11T02:18:55.941 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="9"
2024-03-11T02:18:55.941 INFO:teuthology.orchestra.run.smithi052.stdout:ID="centos"
2024-03-11T02:18:55.941 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE="rhel fedora"
2024-03-11T02:18:55.941 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="9"
2024-03-11T02:18:55.941 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM_ID="platform:el9"
2024-03-11T02:18:55.941 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-11T02:18:55.941 INFO:teuthology.orchestra.run.smithi052.stdout:ANSI_COLOR="0;31"
2024-03-11T02:18:55.941 INFO:teuthology.orchestra.run.smithi052.stdout:LOGO="fedora-logo-icon"
2024-03-11T02:18:55.941 INFO:teuthology.orchestra.run.smithi052.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-11T02:18:55.941 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://centos.org/"
2024-03-11T02:18:55.942 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-11T02:18:55.942 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-11T02:18:55.942 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-11T02:18:55.942 INFO:teuthology.provision.fog.smithi052:Deploy complete!
2024-03-11T02:18:55.943 DEBUG:teuthology.parallel:result is None
2024-03-11T02:18:55.980 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream"
2024-03-11T02:18:55.981 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="9"
2024-03-11T02:18:55.981 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos"
2024-03-11T02:18:55.981 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora"
2024-03-11T02:18:55.981 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="9"
2024-03-11T02:18:55.981 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el9"
2024-03-11T02:18:55.981 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-11T02:18:55.981 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31"
2024-03-11T02:18:55.981 INFO:teuthology.orchestra.run.smithi002.stdout:LOGO="fedora-logo-icon"
2024-03-11T02:18:55.981 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-11T02:18:55.981 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/"
2024-03-11T02:18:55.981 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-11T02:18:55.981 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-11T02:18:55.981 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-11T02:18:55.982 INFO:teuthology.provision.fog.smithi002:Deploy complete!
2024-03-11T02:18:55.982 DEBUG:teuthology.parallel:result is None
2024-03-11T02:18:56.242 INFO:teuthology.lock.ops:Checking smithi002.front.sepia.ceph.com
2024-03-11T02:18:56.242 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-11T02:18:56.268 INFO:teuthology.lock.ops:Checking smithi033.front.sepia.ceph.com
2024-03-11T02:18:56.268 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-11T02:18:56.293 INFO:teuthology.lock.ops:Checking smithi052.front.sepia.ceph.com
2024-03-11T02:18:56.293 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-11T02:18:56.312 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: set os type and version on server
2024-03-11T02:18:56.312 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com'
2024-03-11T02:18:56.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:18:56.412 DEBUG:teuthology.orchestra.run.smithi002:> true
2024-03-11T02:18:56.476 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com'
2024-03-11T02:18:56.477 DEBUG:teuthology.orchestra.run.smithi002:> uname -m
2024-03-11T02:18:56.532 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64
2024-03-11T02:18:56.532 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release
2024-03-11T02:18:56.586 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream"
2024-03-11T02:18:56.586 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="9"
2024-03-11T02:18:56.587 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos"
2024-03-11T02:18:56.587 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora"
2024-03-11T02:18:56.587 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="9"
2024-03-11T02:18:56.587 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el9"
2024-03-11T02:18:56.587 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-11T02:18:56.587 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31"
2024-03-11T02:18:56.587 INFO:teuthology.orchestra.run.smithi002.stdout:LOGO="fedora-logo-icon"
2024-03-11T02:18:56.587 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-11T02:18:56.587 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/"
2024-03-11T02:18:56.587 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-11T02:18:56.587 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-11T02:18:56.587 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-11T02:18:56.588 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server
2024-03-11T02:18:56.610 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: set os type and version on server
2024-03-11T02:18:56.610 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com'
2024-03-11T02:18:56.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:18:56.715 DEBUG:teuthology.orchestra.run.smithi033:> true
2024-03-11T02:18:56.787 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com'
2024-03-11T02:18:56.787 DEBUG:teuthology.orchestra.run.smithi033:> uname -m
2024-03-11T02:18:56.844 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64
2024-03-11T02:18:56.844 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release
2024-03-11T02:18:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream"
2024-03-11T02:18:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9"
2024-03-11T02:18:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos"
2024-03-11T02:18:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora"
2024-03-11T02:18:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9"
2024-03-11T02:18:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9"
2024-03-11T02:18:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-11T02:18:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31"
2024-03-11T02:18:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon"
2024-03-11T02:18:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-11T02:18:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/"
2024-03-11T02:18:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-11T02:18:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-11T02:18:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-11T02:18:56.904 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server
2024-03-11T02:18:56.935 INFO:teuthology.lock.ops:Updating [smithi052.front.sepia.ceph.com]: set os type and version on server
2024-03-11T02:18:56.935 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2024-03-11T02:18:56.936 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T02:18:57.040 DEBUG:teuthology.orchestra.run.smithi052:> true
2024-03-11T02:18:57.111 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com'
2024-03-11T02:18:57.111 DEBUG:teuthology.orchestra.run.smithi052:> uname -m
2024-03-11T02:18:57.167 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64
2024-03-11T02:18:57.168 DEBUG:teuthology.orchestra.run.smithi052:> cat /etc/os-release
2024-03-11T02:18:57.224 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="CentOS Stream"
2024-03-11T02:18:57.224 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="9"
2024-03-11T02:18:57.224 INFO:teuthology.orchestra.run.smithi052.stdout:ID="centos"
2024-03-11T02:18:57.225 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE="rhel fedora"
2024-03-11T02:18:57.225 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="9"
2024-03-11T02:18:57.225 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM_ID="platform:el9"
2024-03-11T02:18:57.225 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-11T02:18:57.225 INFO:teuthology.orchestra.run.smithi052.stdout:ANSI_COLOR="0;31"
2024-03-11T02:18:57.225 INFO:teuthology.orchestra.run.smithi052.stdout:LOGO="fedora-logo-icon"
2024-03-11T02:18:57.225 INFO:teuthology.orchestra.run.smithi052.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-11T02:18:57.225 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://centos.org/"
2024-03-11T02:18:57.225 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-11T02:18:57.225 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-11T02:18:57.225 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-11T02:18:57.226 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server
2024-03-11T02:18:57.244 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:18:57.352 INFO:teuthology.dispatcher.supervisor:Running job 7590292
2024-03-11T02:18:57.353 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-03-11_01:40:49-fs-main-testing-default-smithi/7590292 --name vshankar-2024-03-11_01:40:49-fs-main-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} -- /home/teuthworker/archive/vshankar-2024-03-11_01:40:49-fs-main-testing-default-smithi/7590292/orig.config.yaml
2024-03-11T02:18:57.355 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-03-11_01:40:49-fs-main-testing-default-smithi/7590292
2024-03-11T02:18:57.356 INFO:teuthology.dispatcher.supervisor:Job PID: 3278640
2024-03-11T02:18:57.356 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-03-11T02:20:57.357 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:22:57.440 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:24:57.513 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:26:57.588 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:28:57.665 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:30:57.745 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:32:57.825 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:34:57.898 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:36:57.973 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:38:58.040 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:40:58.116 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:42:58.201 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:44:58.274 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:46:58.349 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:48:58.419 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:50:58.475 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:52:58.562 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:54:58.630 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:56:58.702 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T02:58:58.773 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T03:00:58.846 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T03:02:58.912 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T03:04:58.986 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T03:06:59.059 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T03:08:59.130 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T03:10:59.206 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T03:12:59.295 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T03:14:59.362 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T03:16:59.435 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T03:18:59.534 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T03:20:59.611 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T03:22:59.684 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T03:24:59.752 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T03:26:59.831 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T03:28:59.900 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T03:28:59.981 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1