2024-04-01T02:57:50.566 INFO:root:teuthology version: 0.0.1.dev262+ge691533
2024-04-01T02:57:50.566 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-01T02:57:50.733 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fe875827f40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fe876888c10>, signals=[15])
2024-04-01T02:57:50.733 INFO:teuthology.lock.ops:Start node 'smithi098.front.sepia.ceph.com' reimaging
2024-04-01T02:57:50.733 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: reset os type and version on server
2024-04-01T02:57:50.733 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server
2024-04-01T02:57:50.758 INFO:teuthology.lock.ops:Start node 'smithi111.front.sepia.ceph.com' reimaging
2024-04-01T02:57:50.758 INFO:teuthology.lock.ops:Updating [smithi111.front.sepia.ceph.com]: reset os type and version on server
2024-04-01T02:57:50.758 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server
2024-04-01T02:57:50.783 INFO:teuthology.lock.ops:Start node 'smithi165.front.sepia.ceph.com' reimaging
2024-04-01T02:57:50.783 INFO:teuthology.lock.ops:Updating [smithi165.front.sepia.ceph.com]: reset os type and version on server
2024-04-01T02:57:50.783 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server
2024-04-01T02:57:50.873 DEBUG:teuthology.provision.fog.smithi098:Requesting image smithi_centos_9.stream (ID 10)
2024-04-01T02:57:50.901 DEBUG:teuthology.provision.fog.smithi111:Requesting image smithi_centos_9.stream (ID 10)
2024-04-01T02:57:50.922 INFO:teuthology.provision.fog.smithi098:Scheduling deploy of centos 9.stream
2024-04-01T02:57:50.926 DEBUG:teuthology.provision.fog.smithi165:Requesting image smithi_centos_9.stream (ID 10)
2024-04-01T02:57:50.954 INFO:teuthology.provision.fog.smithi111:Scheduling deploy of centos 9.stream
2024-04-01T02:57:50.970 INFO:teuthology.provision.fog.smithi165:Scheduling deploy of centos 9.stream
2024-04-01T02:57:51.200 INFO:teuthology.orchestra.console.smithi098:Power off
2024-04-01T02:57:51.200 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-01T02:57:51.214 INFO:teuthology.orchestra.console.smithi111:Power off
2024-04-01T02:57:51.214 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-01T02:57:51.226 INFO:teuthology.orchestra.console.smithi165:Power off
2024-04-01T02:57:51.226 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-01T02:57:51.235 DEBUG:teuthology.orchestra.console.smithi098:power off output: Chassis Power Control: Down/Off
2024-04-01T02:57:51.235 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:57:51.246 DEBUG:teuthology.orchestra.console.smithi111:power off output: Chassis Power Control: Down/Off
2024-04-01T02:57:51.247 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:57:51.255 DEBUG:teuthology.orchestra.console.smithi165:power off output: Chassis Power Control: Down/Off
2024-04-01T02:57:51.256 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:57:51.265 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on
2024-04-01T02:57:51.269 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on
2024-04-01T02:57:51.278 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is on
2024-04-01T02:57:55.266 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:57:55.274 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:57:55.280 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:57:55.376 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on
2024-04-01T02:57:55.382 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on
2024-04-01T02:57:55.388 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is on
2024-04-01T02:57:59.378 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:57:59.385 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:57:59.391 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:57:59.487 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on
2024-04-01T02:57:59.493 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is off
2024-04-01T02:57:59.500 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is on
2024-04-01T02:57:59.594 INFO:teuthology.orchestra.console.smithi111:Power off completed
2024-04-01T02:57:59.695 INFO:teuthology.orchestra.console.smithi111:Power on
2024-04-01T02:57:59.696 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-01T02:57:59.717 DEBUG:teuthology.orchestra.console.smithi111:power on output: Chassis Power Control: Up/On
2024-04-01T02:57:59.717 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:57:59.740 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is off
2024-04-01T02:58:03.490 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:58:03.501 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:58:03.599 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off
2024-04-01T02:58:03.609 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is off
2024-04-01T02:58:03.700 INFO:teuthology.orchestra.console.smithi098:Power off completed
2024-04-01T02:58:03.710 INFO:teuthology.orchestra.console.smithi165:Power off completed
2024-04-01T02:58:03.740 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:58:03.801 INFO:teuthology.orchestra.console.smithi098:Power on
2024-04-01T02:58:03.802 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-01T02:58:03.811 INFO:teuthology.orchestra.console.smithi165:Power on
2024-04-01T02:58:03.811 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-01T02:58:03.823 DEBUG:teuthology.orchestra.console.smithi098:power on output: Chassis Power Control: Up/On
2024-04-01T02:58:03.824 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:58:03.834 DEBUG:teuthology.orchestra.console.smithi165:power on output: Chassis Power Control: Up/On
2024-04-01T02:58:03.834 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:58:03.846 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off
2024-04-01T02:58:03.850 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is off
2024-04-01T02:58:03.856 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is off
2024-04-01T02:58:07.850 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:58:07.857 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:58:07.864 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:58:07.959 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off
2024-04-01T02:58:07.966 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on
2024-04-01T02:58:07.974 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is off
2024-04-01T02:58:08.067 INFO:teuthology.orchestra.console.smithi111:Power on completed
2024-04-01T02:58:08.168 INFO:teuthology.provision.fog.smithi111:Waiting for deploy to finish
2024-04-01T02:58:11.964 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:58:11.975 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-01T02:58:12.073 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on
2024-04-01T02:58:12.083 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is on
2024-04-01T02:58:12.174 INFO:teuthology.orchestra.console.smithi098:Power on completed
2024-04-01T02:58:12.184 INFO:teuthology.orchestra.console.smithi165:Power on completed
2024-04-01T02:58:12.275 INFO:teuthology.provision.fog.smithi098:Waiting for deploy to finish
2024-04-01T02:58:12.285 INFO:teuthology.provision.fog.smithi165:Waiting for deploy to finish
2024-04-01T03:00:43.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:00:46.209 WARNING:teuthology.provision.fog.smithi165:[Errno None] Unable to connect to port 22 on 172.21.15.165
2024-04-01T03:00:52.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:00:55.294 WARNING:teuthology.provision.fog.smithi165:[Errno None] Unable to connect to port 22 on 172.21.15.165
2024-04-01T03:00:58.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:01.278 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2024-04-01T03:01:01.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:04.350 WARNING:teuthology.provision.fog.smithi165:[Errno None] Unable to connect to port 22 on 172.21.15.165
2024-04-01T03:01:07.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:09.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:10.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:10.462 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2024-04-01T03:01:12.350 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111
2024-04-01T03:01:13.406 WARNING:teuthology.provision.fog.smithi165:[Errno None] Unable to connect to port 22 on 172.21.15.165
2024-04-01T03:01:16.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:18.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:19.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:19.522 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2024-04-01T03:01:21.406 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111
2024-04-01T03:01:22.462 WARNING:teuthology.provision.fog.smithi165:[Errno None] Unable to connect to port 22 on 172.21.15.165
2024-04-01T03:01:25.526 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:27.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:28.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:28.606 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2024-04-01T03:01:30.466 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111
2024-04-01T03:01:31.518 WARNING:teuthology.provision.fog.smithi165:[Errno None] Unable to connect to port 22 on 172.21.15.165
2024-04-01T03:01:34.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:36.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:37.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:37.665 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2024-04-01T03:01:39.554 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111
2024-04-01T03:01:40.574 WARNING:teuthology.provision.fog.smithi165:[Errno None] Unable to connect to port 22 on 172.21.15.165
2024-04-01T03:01:43.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:45.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:46.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:46.718 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2024-04-01T03:01:48.638 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111
2024-04-01T03:01:52.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:54.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:01:57.698 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111
2024-04-01T03:02:03.701 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:02:06.786 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111
2024-04-01T03:02:12.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:02:46.611 WARNING:teuthology.provision.fog.smithi165:timed out
2024-04-01T03:02:52.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:02:52.725 WARNING:teuthology.provision.fog.smithi098:timed out
2024-04-01T03:02:52.844 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-01T03:02:52.845 DEBUG:teuthology.orchestra.run.smithi165:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-01T03:02:53.169 INFO:teuthology.provision.fog.smithi165:Node is ready
2024-04-01T03:02:53.169 DEBUG:teuthology.orchestra.run.smithi165:> hostname
2024-04-01T03:02:53.184 INFO:teuthology.orchestra.run.smithi165.stdout:smithi165.front.sepia.ceph.com
2024-04-01T03:02:53.185 DEBUG:teuthology.orchestra.run.smithi165:> grep smithi165.front.sepia.ceph.com /etc/hosts
2024-04-01T03:02:53.240 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.15.165 smithi165.front.sepia.ceph.com smithi165
2024-04-01T03:02:53.240 DEBUG:teuthology.orchestra.run.smithi165:> sudo hostname smithi165
2024-04-01T03:02:53.353 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i -e 's/smithi165.front.sepia.ceph.com/smithi165/g' /etc/hosts
2024-04-01T03:02:53.381 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i -e 's/172.21.15.165/172.21.15.165/g' /etc/hosts
2024-04-01T03:02:53.449 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i -e 's/smithi165.front.sepia.ceph.com/smithi165/g' /etc/hostname
2024-04-01T03:02:53.517 DEBUG:teuthology.orchestra.run.smithi165:> sudo hostname smithi165
2024-04-01T03:02:53.584 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release
2024-04-01T03:02:53.638 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream"
2024-04-01T03:02:53.639 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9"
2024-04-01T03:02:53.639 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos"
2024-04-01T03:02:53.639 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora"
2024-04-01T03:02:53.639 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9"
2024-04-01T03:02:53.639 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9"
2024-04-01T03:02:53.639 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-01T03:02:53.639 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31"
2024-04-01T03:02:53.639 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon"
2024-04-01T03:02:53.639 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-01T03:02:53.639 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/"
2024-04-01T03:02:53.639 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-01T03:02:53.639 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-01T03:02:53.639 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-01T03:02:53.640 INFO:teuthology.provision.fog.smithi165:Deploy complete!
2024-04-01T03:02:53.641 DEBUG:teuthology.parallel:result is None
2024-04-01T03:02:58.730 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:02:58.922 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-01T03:02:58.922 DEBUG:teuthology.orchestra.run.smithi098:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-01T03:02:59.242 INFO:teuthology.provision.fog.smithi098:Node is ready
2024-04-01T03:02:59.242 DEBUG:teuthology.orchestra.run.smithi098:> hostname
2024-04-01T03:02:59.257 INFO:teuthology.orchestra.run.smithi098.stdout:smithi098.front.sepia.ceph.com
2024-04-01T03:02:59.257 DEBUG:teuthology.orchestra.run.smithi098:> grep smithi098.front.sepia.ceph.com /etc/hosts
2024-04-01T03:02:59.312 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98 smithi098.front.sepia.ceph.com smithi098
2024-04-01T03:02:59.312 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname smithi098
2024-04-01T03:02:59.426 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/smithi098.front.sepia.ceph.com/smithi098/g' /etc/hosts
2024-04-01T03:02:59.454 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/172.21.15.98/172.21.15.98/g' /etc/hosts
2024-04-01T03:02:59.522 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/smithi098.front.sepia.ceph.com/smithi098/g' /etc/hostname
2024-04-01T03:02:59.589 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname smithi098
2024-04-01T03:02:59.654 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release
2024-04-01T03:02:59.709 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream"
2024-04-01T03:02:59.709 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="9"
2024-04-01T03:02:59.709 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos"
2024-04-01T03:02:59.709 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora"
2024-04-01T03:02:59.709 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="9"
2024-04-01T03:02:59.709 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el9"
2024-04-01T03:02:59.709 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-01T03:02:59.710 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31"
2024-04-01T03:02:59.710 INFO:teuthology.orchestra.run.smithi098.stdout:LOGO="fedora-logo-icon"
2024-04-01T03:02:59.710 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-01T03:02:59.710 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/"
2024-04-01T03:02:59.710 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-01T03:02:59.710 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-01T03:02:59.710 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-01T03:02:59.710 INFO:teuthology.provision.fog.smithi098:Deploy complete!
2024-04-01T03:02:59.711 DEBUG:teuthology.parallel:result is None
2024-04-01T03:03:12.801 WARNING:teuthology.provision.fog.smithi111:timed out
2024-04-01T03:03:18.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:03:19.071 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-01T03:03:19.072 DEBUG:teuthology.orchestra.run.smithi111:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-01T03:03:19.390 INFO:teuthology.provision.fog.smithi111:Node is ready
2024-04-01T03:03:19.390 DEBUG:teuthology.orchestra.run.smithi111:> hostname
2024-04-01T03:03:19.405 INFO:teuthology.orchestra.run.smithi111.stdout:smithi111.front.sepia.ceph.com
2024-04-01T03:03:19.405 DEBUG:teuthology.orchestra.run.smithi111:> grep smithi111.front.sepia.ceph.com /etc/hosts
2024-04-01T03:03:19.461 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111 smithi111.front.sepia.ceph.com smithi111
2024-04-01T03:03:19.462 DEBUG:teuthology.orchestra.run.smithi111:> sudo hostname smithi111
2024-04-01T03:03:19.553 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i -e 's/smithi111.front.sepia.ceph.com/smithi111/g' /etc/hosts
2024-04-01T03:03:19.583 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i -e 's/172.21.15.111/172.21.15.111/g' /etc/hosts
2024-04-01T03:03:19.649 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i -e 's/smithi111.front.sepia.ceph.com/smithi111/g' /etc/hostname
2024-04-01T03:03:19.716 DEBUG:teuthology.orchestra.run.smithi111:> sudo hostname smithi111
2024-04-01T03:03:19.786 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release
2024-04-01T03:03:19.839 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream"
2024-04-01T03:03:19.840 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="9"
2024-04-01T03:03:19.840 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos"
2024-04-01T03:03:19.840 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora"
2024-04-01T03:03:19.840 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="9"
2024-04-01T03:03:19.840 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el9"
2024-04-01T03:03:19.840 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-01T03:03:19.840 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31"
2024-04-01T03:03:19.840 INFO:teuthology.orchestra.run.smithi111.stdout:LOGO="fedora-logo-icon"
2024-04-01T03:03:19.840 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-01T03:03:19.840 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/"
2024-04-01T03:03:19.840 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-01T03:03:19.841 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-01T03:03:19.841 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-01T03:03:19.841 INFO:teuthology.provision.fog.smithi111:Deploy complete!
2024-04-01T03:03:19.841 DEBUG:teuthology.parallel:result is None
2024-04-01T03:03:20.050 INFO:teuthology.lock.ops:Checking smithi098.front.sepia.ceph.com
2024-04-01T03:03:20.050 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-01T03:03:20.071 INFO:teuthology.lock.ops:Checking smithi111.front.sepia.ceph.com
2024-04-01T03:03:20.071 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-01T03:03:20.087 INFO:teuthology.lock.ops:Checking smithi165.front.sepia.ceph.com
2024-04-01T03:03:20.087 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-01T03:03:20.105 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: set os type and version on server
2024-04-01T03:03:20.106 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2024-04-01T03:03:20.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:03:20.198 DEBUG:teuthology.orchestra.run.smithi098:> true
2024-04-01T03:03:20.371 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com'
2024-04-01T03:03:20.517 DEBUG:teuthology.orchestra.run.smithi098:> uname -m
2024-04-01T03:03:20.532 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64
2024-04-01T03:03:20.532 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release
2024-04-01T03:03:20.587 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream"
2024-04-01T03:03:20.587 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="9"
2024-04-01T03:03:20.587 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos"
2024-04-01T03:03:20.587 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora"
2024-04-01T03:03:20.587 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="9"
2024-04-01T03:03:20.587 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el9"
2024-04-01T03:03:20.587 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-01T03:03:20.588 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31"
2024-04-01T03:03:20.588 INFO:teuthology.orchestra.run.smithi098.stdout:LOGO="fedora-logo-icon"
2024-04-01T03:03:20.588 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-01T03:03:20.588 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/"
2024-04-01T03:03:20.588 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-01T03:03:20.588 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-01T03:03:20.588 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-01T03:03:20.589 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server
2024-04-01T03:03:20.613 INFO:teuthology.lock.ops:Updating [smithi111.front.sepia.ceph.com]: set os type and version on server
2024-04-01T03:03:20.613 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com'
2024-04-01T03:03:20.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:03:20.699 DEBUG:teuthology.orchestra.run.smithi111:> true
2024-04-01T03:03:20.764 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com'
2024-04-01T03:03:20.765 DEBUG:teuthology.orchestra.run.smithi111:> uname -m
2024-04-01T03:03:20.819 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64
2024-04-01T03:03:20.819 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release
2024-04-01T03:03:20.874 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream"
2024-04-01T03:03:20.874 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="9"
2024-04-01T03:03:20.874 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos"
2024-04-01T03:03:20.874 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora"
2024-04-01T03:03:20.874 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="9"
2024-04-01T03:03:20.874 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el9"
2024-04-01T03:03:20.874 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-01T03:03:20.874 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31"
2024-04-01T03:03:20.874 INFO:teuthology.orchestra.run.smithi111.stdout:LOGO="fedora-logo-icon"
2024-04-01T03:03:20.875 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-01T03:03:20.875 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/"
2024-04-01T03:03:20.875 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-01T03:03:20.875 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-01T03:03:20.875 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-01T03:03:20.875 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server
2024-04-01T03:03:20.896 INFO:teuthology.lock.ops:Updating [smithi165.front.sepia.ceph.com]: set os type and version on server
2024-04-01T03:03:20.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com'
2024-04-01T03:03:20.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-01T03:03:20.988 DEBUG:teuthology.orchestra.run.smithi165:> true
2024-04-01T03:03:21.156 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com'
2024-04-01T03:03:21.157 DEBUG:teuthology.orchestra.run.smithi165:> uname -m
2024-04-01T03:03:21.211 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64
2024-04-01T03:03:21.211 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release
2024-04-01T03:03:21.266 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream"
2024-04-01T03:03:21.266 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9"
2024-04-01T03:03:21.267 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos"
2024-04-01T03:03:21.267 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora"
2024-04-01T03:03:21.267 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9"
2024-04-01T03:03:21.267 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9"
2024-04-01T03:03:21.267 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-01T03:03:21.267 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31"
2024-04-01T03:03:21.267 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon"
2024-04-01T03:03:21.267 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-01T03:03:21.267 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/"
2024-04-01T03:03:21.267 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-01T03:03:21.267 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-01T03:03:21.267 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-01T03:03:21.268 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server
2024-04-01T03:03:21.287 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-01T03:03:21.399 INFO:teuthology.dispatcher.supervisor:Running job 7633321
2024-04-01T03:03:21.399 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633321 --name teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/pjd}} -- /home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633321/orig.config.yaml
2024-04-01T03:03:21.402 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633321
2024-04-01T03:03:21.402 INFO:teuthology.dispatcher.supervisor:Job PID: 3760705
2024-04-01T03:03:21.403 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-04-01T03:05:21.502 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-01T03:07:21.674 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-01T03:09:21.840 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-01T03:11:21.995 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-01T03:13:22.168 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-01T03:15:22.335 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-01T03:17:22.509 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-01T03:19:22.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-01T03:21:22.891 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-01T03:23:23.067 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-01T03:25:23.240 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-01T03:27:23.404 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-01T03:29:23.591 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-01T03:31:23.767 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-01T03:31:23.841 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1