2024-02-26T04:26:38.460 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55
2024-02-26T04:26:38.460 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:26:38.664 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f500b69ff40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f500c713c10>, signals=[15])
2024-02-26T04:26:38.664 INFO:teuthology.lock.ops:Start node 'smithi023.front.sepia.ceph.com' reimaging
2024-02-26T04:26:38.664 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: reset os type and version on server
2024-02-26T04:26:38.665 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server
2024-02-26T04:26:39.956 INFO:teuthology.lock.ops:Start node 'smithi044.front.sepia.ceph.com' reimaging
2024-02-26T04:26:39.956 INFO:teuthology.lock.ops:Updating [smithi044.front.sepia.ceph.com]: reset os type and version on server
2024-02-26T04:26:39.956 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server
2024-02-26T04:26:39.980 INFO:teuthology.lock.ops:Start node 'smithi158.front.sepia.ceph.com' reimaging
2024-02-26T04:26:39.980 INFO:teuthology.lock.ops:Updating [smithi158.front.sepia.ceph.com]: reset os type and version on server
2024-02-26T04:26:39.980 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server
2024-02-26T04:26:40.443 DEBUG:teuthology.provision.fog.smithi023:Requesting image smithi_centos_9.stream (ID 10)
2024-02-26T04:26:40.482 DEBUG:teuthology.provision.fog.smithi044:Requesting image smithi_centos_9.stream (ID 10)
2024-02-26T04:26:40.490 DEBUG:teuthology.provision.fog.smithi158:Requesting image smithi_centos_9.stream (ID 10)
2024-02-26T04:26:40.575 INFO:teuthology.provision.fog.smithi023:Scheduling deploy of centos 9.stream
2024-02-26T04:26:40.619 INFO:teuthology.provision.fog.smithi044:Scheduling deploy of centos 9.stream
2024-02-26T04:26:40.682 INFO:teuthology.provision.fog.smithi158:Scheduling deploy of centos 9.stream
2024-02-26T04:26:41.592 INFO:teuthology.orchestra.console.smithi023:Power off
2024-02-26T04:26:41.592 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-02-26T04:26:41.614 DEBUG:teuthology.orchestra.console.smithi023:power off output: Chassis Power Control: Down/Off
2024-02-26T04:26:41.615 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:41.638 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on
2024-02-26T04:26:41.725 INFO:teuthology.orchestra.console.smithi044:Power off
2024-02-26T04:26:41.726 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-02-26T04:26:41.737 INFO:teuthology.orchestra.console.smithi158:Power off
2024-02-26T04:26:41.737 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-02-26T04:26:41.749 DEBUG:teuthology.orchestra.console.smithi044:power off output: Chassis Power Control: Down/Off
2024-02-26T04:26:41.749 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:41.759 DEBUG:teuthology.orchestra.console.smithi158:power off output: Chassis Power Control: Down/Off
2024-02-26T04:26:41.759 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:41.770 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on
2024-02-26T04:26:41.781 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on
2024-02-26T04:26:45.644 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:45.753 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on
2024-02-26T04:26:45.772 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:45.783 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:45.881 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on
2024-02-26T04:26:45.891 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on
2024-02-26T04:26:49.759 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:49.867 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off
2024-02-26T04:26:49.882 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:49.892 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:49.969 INFO:teuthology.orchestra.console.smithi023:Power off completed
2024-02-26T04:26:49.991 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on
2024-02-26T04:26:50.003 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on
2024-02-26T04:26:50.070 INFO:teuthology.orchestra.console.smithi023:Power on
2024-02-26T04:26:50.070 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-02-26T04:26:50.091 DEBUG:teuthology.orchestra.console.smithi023:power on output: Chassis Power Control: Up/On
2024-02-26T04:26:50.091 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:50.112 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off
2024-02-26T04:26:53.996 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:54.003 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:54.107 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off
2024-02-26T04:26:54.111 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off
2024-02-26T04:26:54.112 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:54.208 INFO:teuthology.orchestra.console.smithi044:Power off completed
2024-02-26T04:26:54.211 INFO:teuthology.orchestra.console.smithi158:Power off completed
2024-02-26T04:26:54.221 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off
2024-02-26T04:26:54.309 INFO:teuthology.orchestra.console.smithi044:Power on
2024-02-26T04:26:54.309 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-02-26T04:26:54.316 INFO:teuthology.orchestra.console.smithi158:Power on
2024-02-26T04:26:54.316 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-02-26T04:26:54.330 DEBUG:teuthology.orchestra.console.smithi044:power on output: Chassis Power Control: Up/On
2024-02-26T04:26:54.330 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:54.336 DEBUG:teuthology.orchestra.console.smithi158:power on output: Chassis Power Control: Up/On
2024-02-26T04:26:54.337 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:54.350 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off
2024-02-26T04:26:54.357 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off
2024-02-26T04:26:58.224 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:58.332 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on
2024-02-26T04:26:58.351 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:58.358 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-02-26T04:26:58.434 INFO:teuthology.orchestra.console.smithi023:Power on completed
2024-02-26T04:26:58.459 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on
2024-02-26T04:26:58.466 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on
2024-02-26T04:26:58.535 INFO:teuthology.provision.fog.smithi023:Waiting for deploy to finish
2024-02-26T04:26:58.560 INFO:teuthology.orchestra.console.smithi044:Power on completed
2024-02-26T04:26:58.566 INFO:teuthology.orchestra.console.smithi158:Power on completed
2024-02-26T04:26:58.661 INFO:teuthology.provision.fog.smithi044:Waiting for deploy to finish
2024-02-26T04:26:58.668 INFO:teuthology.provision.fog.smithi158:Waiting for deploy to finish
2024-02-26T04:32:41.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:32:44.444 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44
2024-02-26T04:32:50.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:32:53.504 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44
2024-02-26T04:32:56.356 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:32:56.442 ERROR:teuthology.orchestra.connection:Error authenticating with smithi023.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-02-26T04:32:59.505 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:33:02.556 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44
2024-02-26T04:33:08.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:33:11.616 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44
2024-02-26T04:33:17.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:33:20.668 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44
2024-02-26T04:33:26.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:33:29.724 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44
2024-02-26T04:33:35.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:33:56.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:00.028 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2024-02-26T04:34:00.445 WARNING:teuthology.provision.fog.smithi023:timed out
2024-02-26T04:34:06.029 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:06.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:09.084 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2024-02-26T04:34:09.500 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2024-02-26T04:34:15.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:15.502 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:18.140 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2024-02-26T04:34:18.556 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2024-02-26T04:34:24.141 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:24.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:27.197 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2024-02-26T04:34:27.612 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2024-02-26T04:34:33.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:33.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:35.726 WARNING:teuthology.provision.fog.smithi044:timed out
2024-02-26T04:34:36.252 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2024-02-26T04:34:41.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:42.007 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-02-26T04:34:42.008 DEBUG:teuthology.orchestra.run.smithi044:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-02-26T04:34:42.253 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:42.327 INFO:teuthology.provision.fog.smithi044:Node is ready
2024-02-26T04:34:42.327 DEBUG:teuthology.orchestra.run.smithi044:> hostname
2024-02-26T04:34:42.342 INFO:teuthology.orchestra.run.smithi044.stdout:smithi044.front.sepia.ceph.com
2024-02-26T04:34:42.343 DEBUG:teuthology.orchestra.run.smithi044:> grep smithi044.front.sepia.ceph.com /etc/hosts
2024-02-26T04:34:42.398 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44 smithi044.front.sepia.ceph.com smithi044
2024-02-26T04:34:42.398 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname smithi044
2024-02-26T04:34:42.528 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/smithi044.front.sepia.ceph.com/smithi044/g' /etc/hosts
2024-02-26T04:34:42.560 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/172.21.15.44/172.21.15.44/g' /etc/hosts
2024-02-26T04:34:42.631 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/smithi044.front.sepia.ceph.com/smithi044/g' /etc/hostname
2024-02-26T04:34:42.702 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname smithi044
2024-02-26T04:34:42.772 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release
2024-02-26T04:34:42.827 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream"
2024-02-26T04:34:42.827 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="9"
2024-02-26T04:34:42.827 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos"
2024-02-26T04:34:42.827 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora"
2024-02-26T04:34:42.828 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="9"
2024-02-26T04:34:42.828 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el9"
2024-02-26T04:34:42.828 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-26T04:34:42.828 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31"
2024-02-26T04:34:42.828 INFO:teuthology.orchestra.run.smithi044.stdout:LOGO="fedora-logo-icon"
2024-02-26T04:34:42.828 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-26T04:34:42.828 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/"
2024-02-26T04:34:42.828 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-26T04:34:42.828 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-26T04:34:42.828 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-26T04:34:42.829 INFO:teuthology.provision.fog.smithi044:Deploy complete!
2024-02-26T04:34:42.850 DEBUG:teuthology.parallel:result is None
2024-02-26T04:34:45.308 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2024-02-26T04:34:51.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:34:54.364 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2024-02-26T04:35:00.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:35:03.420 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2024-02-26T04:35:09.421 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:35:27.932 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2024-02-26T04:35:33.615 WARNING:teuthology.provision.fog.smithi023:timed out
2024-02-26T04:35:33.933 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:35:36.988 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2024-02-26T04:35:39.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:35:39.927 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-02-26T04:35:39.928 DEBUG:teuthology.orchestra.run.smithi023:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-02-26T04:35:40.255 INFO:teuthology.provision.fog.smithi023:Node is ready
2024-02-26T04:35:40.255 DEBUG:teuthology.orchestra.run.smithi023:> hostname
2024-02-26T04:35:40.269 INFO:teuthology.orchestra.run.smithi023.stdout:smithi023.front.sepia.ceph.com
2024-02-26T04:35:40.270 DEBUG:teuthology.orchestra.run.smithi023:> grep smithi023.front.sepia.ceph.com /etc/hosts
2024-02-26T04:35:40.324 INFO:teuthology.orchestra.run.smithi023.stdout:172.21.15.23 smithi023.front.sepia.ceph.com smithi023
2024-02-26T04:35:40.325 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023
2024-02-26T04:35:40.437 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hosts
2024-02-26T04:35:40.468 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/172.21.15.23/172.21.15.23/g' /etc/hosts
2024-02-26T04:35:40.538 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hostname
2024-02-26T04:35:40.609 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023
2024-02-26T04:35:40.679 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release
2024-02-26T04:35:40.734 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Stream"
2024-02-26T04:35:40.734 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="9"
2024-02-26T04:35:40.734 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos"
2024-02-26T04:35:40.734 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora"
2024-02-26T04:35:40.734 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="9"
2024-02-26T04:35:40.734 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el9"
2024-02-26T04:35:40.734 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-26T04:35:40.734 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31"
2024-02-26T04:35:40.734 INFO:teuthology.orchestra.run.smithi023.stdout:LOGO="fedora-logo-icon"
2024-02-26T04:35:40.734 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-26T04:35:40.734 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://centos.org/"
2024-02-26T04:35:40.734 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-26T04:35:40.735 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-26T04:35:40.735 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-26T04:35:40.735 INFO:teuthology.provision.fog.smithi023:Deploy complete!
2024-02-26T04:35:40.735 DEBUG:teuthology.parallel:result is None
2024-02-26T04:35:42.990 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:35:46.044 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158
2024-02-26T04:35:52.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:35:52.288 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-02-26T04:35:52.288 DEBUG:teuthology.orchestra.run.smithi158:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-02-26T04:36:07.630 INFO:teuthology.provision.fog.smithi158:Node is ready
2024-02-26T04:36:07.631 DEBUG:teuthology.orchestra.run.smithi158:> hostname
2024-02-26T04:36:07.648 INFO:teuthology.orchestra.run.smithi158.stdout:smithi046
2024-02-26T04:36:07.649 DEBUG:teuthology.orchestra.run.smithi158:> grep smithi046 /etc/hosts
2024-02-26T04:36:07.706 INFO:teuthology.orchestra.run.smithi158.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046
2024-02-26T04:36:07.706 DEBUG:teuthology.orchestra.run.smithi158:> sudo hostname smithi158
2024-02-26T04:36:07.881 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/smithi046/smithi158/g' /etc/hosts
2024-02-26T04:36:07.913 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/172.21.15.46/172.21.15.158/g' /etc/hosts
2024-02-26T04:36:07.986 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/smithi046/smithi158/g' /etc/hostname
2024-02-26T04:36:08.056 DEBUG:teuthology.orchestra.run.smithi158:> sudo hostname smithi158
2024-02-26T04:36:08.125 DEBUG:teuthology.orchestra.run.smithi158:> cat /etc/os-release
2024-02-26T04:36:08.180 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="CentOS Stream"
2024-02-26T04:36:08.181 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="9"
2024-02-26T04:36:08.181 INFO:teuthology.orchestra.run.smithi158.stdout:ID="centos"
2024-02-26T04:36:08.181 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="rhel fedora"
2024-02-26T04:36:08.181 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="9"
2024-02-26T04:36:08.181 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el9"
2024-02-26T04:36:08.181 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-26T04:36:08.181 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31"
2024-02-26T04:36:08.181 INFO:teuthology.orchestra.run.smithi158.stdout:LOGO="fedora-logo-icon"
2024-02-26T04:36:08.181 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-26T04:36:08.181 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://centos.org/"
2024-02-26T04:36:08.181 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-26T04:36:08.181 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-26T04:36:08.181 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-26T04:36:08.182 INFO:teuthology.provision.fog.smithi158:Deploy complete!
2024-02-26T04:36:08.182 DEBUG:teuthology.parallel:result is None
2024-02-26T04:36:08.437 INFO:teuthology.lock.ops:Checking smithi023.front.sepia.ceph.com
2024-02-26T04:36:08.437 INFO:teuthology.lock.ops:New key found. Updating...
2024-02-26T04:36:08.460 INFO:teuthology.lock.ops:Checking smithi044.front.sepia.ceph.com
2024-02-26T04:36:08.461 INFO:teuthology.lock.ops:New key found. Updating...
2024-02-26T04:36:08.485 INFO:teuthology.lock.ops:Checking smithi158.front.sepia.ceph.com
2024-02-26T04:36:08.485 INFO:teuthology.lock.ops:New key found. Updating...
2024-02-26T04:36:08.503 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: set os type and version on server
2024-02-26T04:36:08.503 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com'
2024-02-26T04:36:08.504 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:36:08.651 DEBUG:teuthology.orchestra.run.smithi023:> true
2024-02-26T04:36:08.834 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi023.front.sepia.ceph.com'
2024-02-26T04:36:08.834 DEBUG:teuthology.orchestra.run.smithi023:> uname -m
2024-02-26T04:36:08.889 INFO:teuthology.orchestra.run.smithi023.stdout:x86_64
2024-02-26T04:36:08.890 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release
2024-02-26T04:36:08.944 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Stream"
2024-02-26T04:36:08.944 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="9"
2024-02-26T04:36:08.944 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos"
2024-02-26T04:36:08.944 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora"
2024-02-26T04:36:08.944 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="9"
2024-02-26T04:36:08.944 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el9"
2024-02-26T04:36:08.944 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-26T04:36:08.944 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31"
2024-02-26T04:36:08.945 INFO:teuthology.orchestra.run.smithi023.stdout:LOGO="fedora-logo-icon"
2024-02-26T04:36:08.945 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-26T04:36:08.945 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://centos.org/"
2024-02-26T04:36:08.945 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-26T04:36:08.945 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-26T04:36:08.945 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-26T04:36:08.945 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server
2024-02-26T04:36:08.968 INFO:teuthology.lock.ops:Updating [smithi044.front.sepia.ceph.com]: set os type and version on server
2024-02-26T04:36:08.969 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com'
2024-02-26T04:36:08.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:36:09.072 DEBUG:teuthology.orchestra.run.smithi044:> true
2024-02-26T04:36:09.254 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com'
2024-02-26T04:36:09.255 DEBUG:teuthology.orchestra.run.smithi044:> uname -m
2024-02-26T04:36:09.310 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64
2024-02-26T04:36:09.310 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release
2024-02-26T04:36:09.365 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream"
2024-02-26T04:36:09.365 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="9"
2024-02-26T04:36:09.365 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos"
2024-02-26T04:36:09.366 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora"
2024-02-26T04:36:09.366 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="9"
2024-02-26T04:36:09.366 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el9"
2024-02-26T04:36:09.366 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-26T04:36:09.366 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31"
2024-02-26T04:36:09.366 INFO:teuthology.orchestra.run.smithi044.stdout:LOGO="fedora-logo-icon"
2024-02-26T04:36:09.366 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-26T04:36:09.366 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/"
2024-02-26T04:36:09.366 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-26T04:36:09.366 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-26T04:36:09.366 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-26T04:36:09.367 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server
2024-02-26T04:36:09.393 INFO:teuthology.lock.ops:Updating [smithi158.front.sepia.ceph.com]: set os type and version on server
2024-02-26T04:36:09.393 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com'
2024-02-26T04:36:09.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-02-26T04:36:09.487 DEBUG:teuthology.orchestra.run.smithi158:> true
2024-02-26T04:36:09.556 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi158.front.sepia.ceph.com'
2024-02-26T04:36:09.557 DEBUG:teuthology.orchestra.run.smithi158:> uname -m
2024-02-26T04:36:09.613 INFO:teuthology.orchestra.run.smithi158.stdout:x86_64
2024-02-26T04:36:09.613 DEBUG:teuthology.orchestra.run.smithi158:> cat /etc/os-release
2024-02-26T04:36:09.670 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="CentOS Stream"
2024-02-26T04:36:09.670 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="9"
2024-02-26T04:36:09.670 INFO:teuthology.orchestra.run.smithi158.stdout:ID="centos"
2024-02-26T04:36:09.670 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="rhel fedora"
2024-02-26T04:36:09.670 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="9"
2024-02-26T04:36:09.670 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el9"
2024-02-26T04:36:09.670 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="CentOS Stream 9"
2024-02-26T04:36:09.670 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31"
2024-02-26T04:36:09.671 INFO:teuthology.orchestra.run.smithi158.stdout:LOGO="fedora-logo-icon"
2024-02-26T04:36:09.671 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-02-26T04:36:09.671 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://centos.org/"
2024-02-26T04:36:09.671 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-02-26T04:36:09.671 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-02-26T04:36:09.671 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-02-26T04:36:09.671 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server
2024-02-26T04:36:09.693 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:36:09.795 INFO:teuthology.dispatcher.supervisor:Running job 7573361
2024-02-26T04:36:09.796 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573361 --name vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi --description fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/direct_io}} -- /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573361/orig.config.yaml
2024-02-26T04:36:09.799 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-02-26_04:25:56-fs:workload-wip-vshankar-testing-20240216.060239-testing-default-smithi/7573361
2024-02-26T04:36:09.800 INFO:teuthology.dispatcher.supervisor:Job PID: 1028164
2024-02-26T04:36:09.800 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-02-26T04:38:09.801 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:40:09.877 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:42:09.950 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:44:10.020 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:46:10.102 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:48:10.174 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:50:10.247 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:52:10.342 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:54:10.425 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:56:10.503 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T04:58:10.586 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:00:10.743 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:02:10.805 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:04:10.877 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:06:10.951 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:08:11.014 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:10:11.080 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:12:11.160 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:14:11.233 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:16:11.305 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:18:11.379 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:20:11.505 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-02-26T05:20:11.572 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1