2024-03-31T16:00:37.367 INFO:root:teuthology version: 0.0.1.dev262+ge691533
2024-03-31T16:00:37.368 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-31T16:00:37.553 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fd113c7bf40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fd114cefc10>, signals=[15])
2024-03-31T16:00:37.554 INFO:teuthology.lock.ops:Start node 'smithi088.front.sepia.ceph.com' reimaging
2024-03-31T16:00:37.554 INFO:teuthology.lock.ops:Updating [smithi088.front.sepia.ceph.com]: reset os type and version on server
2024-03-31T16:00:37.554 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server
2024-03-31T16:00:37.583 INFO:teuthology.lock.ops:Start node 'smithi099.front.sepia.ceph.com' reimaging
2024-03-31T16:00:37.583 INFO:teuthology.lock.ops:Updating [smithi099.front.sepia.ceph.com]: reset os type and version on server
2024-03-31T16:00:37.583 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server
2024-03-31T16:00:37.607 INFO:teuthology.lock.ops:Start node 'smithi190.front.sepia.ceph.com' reimaging
2024-03-31T16:00:37.607 INFO:teuthology.lock.ops:Updating [smithi190.front.sepia.ceph.com]: reset os type and version on server
2024-03-31T16:00:37.608 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server
2024-03-31T16:00:37.695 DEBUG:teuthology.provision.fog.smithi088:Requesting image smithi_centos_9.stream (ID 10)
2024-03-31T16:00:37.710 DEBUG:teuthology.provision.fog.smithi099:Requesting image smithi_centos_9.stream (ID 10)
2024-03-31T16:00:37.742 DEBUG:teuthology.provision.fog.smithi190:Requesting image smithi_centos_9.stream (ID 10)
2024-03-31T16:00:37.751 INFO:teuthology.provision.fog.smithi088:Scheduling deploy of centos 9.stream
2024-03-31T16:00:37.755 INFO:teuthology.provision.fog.smithi099:Scheduling deploy of centos 9.stream
2024-03-31T16:00:37.788 INFO:teuthology.provision.fog.smithi190:Scheduling deploy of centos 9.stream
2024-03-31T16:00:37.982 INFO:teuthology.orchestra.console.smithi088:Power off
2024-03-31T16:00:37.983 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-31T16:00:37.998 INFO:teuthology.orchestra.console.smithi099:Power off
2024-03-31T16:00:37.998 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-31T16:00:38.009 DEBUG:teuthology.orchestra.console.smithi088:power off output: Chassis Power Control: Down/Off
2024-03-31T16:00:38.010 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:38.021 DEBUG:teuthology.orchestra.console.smithi099:power off output: Chassis Power Control: Down/Off
2024-03-31T16:00:38.022 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:38.032 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on
2024-03-31T16:00:38.037 INFO:teuthology.orchestra.console.smithi190:Power off
2024-03-31T16:00:38.038 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-31T16:00:38.046 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on
2024-03-31T16:00:38.058 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off
2024-03-31T16:00:38.058 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:38.083 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on
2024-03-31T16:00:42.035 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:42.047 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:42.084 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:42.145 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on
2024-03-31T16:00:42.157 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on
2024-03-31T16:00:42.193 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on
2024-03-31T16:00:46.150 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:46.158 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:46.194 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:46.260 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on
2024-03-31T16:00:46.266 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on
2024-03-31T16:00:46.303 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on
2024-03-31T16:00:50.262 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:50.268 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:50.304 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:50.371 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off
2024-03-31T16:00:50.377 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off
2024-03-31T16:00:50.413 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off
2024-03-31T16:00:50.473 INFO:teuthology.orchestra.console.smithi088:Power off completed
2024-03-31T16:00:50.478 INFO:teuthology.orchestra.console.smithi099:Power off completed
2024-03-31T16:00:50.514 INFO:teuthology.orchestra.console.smithi190:Power off completed
2024-03-31T16:00:50.574 INFO:teuthology.orchestra.console.smithi088:Power on
2024-03-31T16:00:50.574 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-31T16:00:50.582 INFO:teuthology.orchestra.console.smithi099:Power on
2024-03-31T16:00:50.583 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-31T16:00:50.597 DEBUG:teuthology.orchestra.console.smithi088:power on output: Chassis Power Control: Up/On
2024-03-31T16:00:50.597 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:50.607 DEBUG:teuthology.orchestra.console.smithi099:power on output: Chassis Power Control: Up/On
2024-03-31T16:00:50.608 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:50.615 INFO:teuthology.orchestra.console.smithi190:Power on
2024-03-31T16:00:50.616 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-31T16:00:50.626 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off
2024-03-31T16:00:50.630 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off
2024-03-31T16:00:50.639 DEBUG:teuthology.orchestra.console.smithi190:power on output: Chassis Power Control: Up/On
2024-03-31T16:00:50.639 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:50.660 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off
2024-03-31T16:00:54.630 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:54.639 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:54.661 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:54.741 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on
2024-03-31T16:00:54.748 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on
2024-03-31T16:00:54.770 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off
2024-03-31T16:00:54.842 INFO:teuthology.orchestra.console.smithi088:Power on completed
2024-03-31T16:00:54.849 INFO:teuthology.orchestra.console.smithi099:Power on completed
2024-03-31T16:00:54.943 INFO:teuthology.provision.fog.smithi088:Waiting for deploy to finish
2024-03-31T16:00:54.950 INFO:teuthology.provision.fog.smithi099:Waiting for deploy to finish
2024-03-31T16:00:58.771 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-31T16:00:58.881 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on
2024-03-31T16:00:58.982 INFO:teuthology.orchestra.console.smithi190:Power on completed
2024-03-31T16:00:59.082 INFO:teuthology.provision.fog.smithi190:Waiting for deploy to finish
2024-03-31T16:03:25.797 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:03:28.862 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-03-31T16:03:29.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:03:32.926 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2024-03-31T16:03:34.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:03:37.918 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-03-31T16:03:38.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:03:41.986 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2024-03-31T16:03:43.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:03:46.974 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-03-31T16:03:47.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:03:51.074 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2024-03-31T16:03:52.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:03:55.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:03:56.030 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-03-31T16:03:57.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:03:59.038 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88
2024-03-31T16:04:00.158 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2024-03-31T16:04:02.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:04:05.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:04:05.086 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-03-31T16:04:06.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:04:08.094 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88
2024-03-31T16:04:09.218 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2024-03-31T16:04:11.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:04:14.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:04:14.142 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-03-31T16:04:15.220 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:04:17.150 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88
2024-03-31T16:04:18.270 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2024-03-31T16:04:20.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:04:23.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:04:24.273 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:04:26.206 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88
2024-03-31T16:04:32.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:04:35.294 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88
2024-03-31T16:04:41.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:04:44.382 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88
2024-03-31T16:04:50.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:04:53.474 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88
2024-03-31T16:04:59.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:05:20.167 WARNING:teuthology.provision.fog.smithi099:timed out
2024-03-31T16:05:24.278 WARNING:teuthology.provision.fog.smithi190:timed out
2024-03-31T16:05:26.169 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:05:26.404 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-31T16:05:26.405 DEBUG:teuthology.orchestra.run.smithi099:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-31T16:05:26.736 INFO:teuthology.provision.fog.smithi099:Node is ready
2024-03-31T16:05:26.737 DEBUG:teuthology.orchestra.run.smithi099:> hostname
2024-03-31T16:05:26.752 INFO:teuthology.orchestra.run.smithi099.stdout:smithi099.front.sepia.ceph.com
2024-03-31T16:05:26.752 DEBUG:teuthology.orchestra.run.smithi099:> grep smithi099.front.sepia.ceph.com /etc/hosts
2024-03-31T16:05:26.807 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99 smithi099.front.sepia.ceph.com smithi099
2024-03-31T16:05:26.807 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname smithi099
2024-03-31T16:05:26.916 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/smithi099.front.sepia.ceph.com/smithi099/g' /etc/hosts
2024-03-31T16:05:26.944 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/172.21.15.99/172.21.15.99/g' /etc/hosts
2024-03-31T16:05:27.011 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/smithi099.front.sepia.ceph.com/smithi099/g' /etc/hostname
2024-03-31T16:05:27.079 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname smithi099
2024-03-31T16:05:27.145 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release
2024-03-31T16:05:27.200 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream"
2024-03-31T16:05:27.200 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9"
2024-03-31T16:05:27.200 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos"
2024-03-31T16:05:27.201 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora"
2024-03-31T16:05:27.201 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9"
2024-03-31T16:05:27.201 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9"
2024-03-31T16:05:27.201 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-31T16:05:27.201 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31"
2024-03-31T16:05:27.201 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon"
2024-03-31T16:05:27.201 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-31T16:05:27.201 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/"
2024-03-31T16:05:27.201 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-31T16:05:27.201 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-31T16:05:27.201 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-31T16:05:27.202 INFO:teuthology.provision.fog.smithi099:Deploy complete!
2024-03-31T16:05:27.204 DEBUG:teuthology.parallel:result is None
2024-03-31T16:05:30.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:05:30.501 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-31T16:05:30.501 DEBUG:teuthology.orchestra.run.smithi190:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-31T16:05:30.825 INFO:teuthology.provision.fog.smithi190:Node is ready
2024-03-31T16:05:30.825 DEBUG:teuthology.orchestra.run.smithi190:> hostname
2024-03-31T16:05:30.839 INFO:teuthology.orchestra.run.smithi190.stdout:smithi190.front.sepia.ceph.com
2024-03-31T16:05:30.839 DEBUG:teuthology.orchestra.run.smithi190:> grep smithi190.front.sepia.ceph.com /etc/hosts
2024-03-31T16:05:30.894 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.15.190 smithi190.front.sepia.ceph.com smithi190
2024-03-31T16:05:30.894 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname smithi190
2024-03-31T16:05:30.993 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i -e 's/smithi190.front.sepia.ceph.com/smithi190/g' /etc/hosts
2024-03-31T16:05:31.020 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i -e 's/172.21.15.190/172.21.15.190/g' /etc/hosts
2024-03-31T16:05:31.087 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i -e 's/smithi190.front.sepia.ceph.com/smithi190/g' /etc/hostname
2024-03-31T16:05:31.153 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname smithi190
2024-03-31T16:05:31.219 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release
2024-03-31T16:05:31.274 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream"
2024-03-31T16:05:31.274 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="9"
2024-03-31T16:05:31.274 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos"
2024-03-31T16:05:31.274 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora"
2024-03-31T16:05:31.274 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="9"
2024-03-31T16:05:31.274 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el9"
2024-03-31T16:05:31.275 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-31T16:05:31.275 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31"
2024-03-31T16:05:31.275 INFO:teuthology.orchestra.run.smithi190.stdout:LOGO="fedora-logo-icon"
2024-03-31T16:05:31.275 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-31T16:05:31.275 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/"
2024-03-31T16:05:31.275 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-31T16:05:31.275 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-31T16:05:31.275 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-31T16:05:31.275 INFO:teuthology.provision.fog.smithi190:Deploy complete!
2024-03-31T16:05:31.276 DEBUG:teuthology.parallel:result is None
2024-03-31T16:05:59.507 WARNING:teuthology.provision.fog.smithi088:timed out
2024-03-31T16:06:05.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:06:05.727 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-31T16:06:05.727 DEBUG:teuthology.orchestra.run.smithi088:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-31T16:06:06.062 INFO:teuthology.provision.fog.smithi088:Node is ready
2024-03-31T16:06:06.062 DEBUG:teuthology.orchestra.run.smithi088:> hostname
2024-03-31T16:06:06.077 INFO:teuthology.orchestra.run.smithi088.stdout:smithi088.front.sepia.ceph.com
2024-03-31T16:06:06.077 DEBUG:teuthology.orchestra.run.smithi088:> grep smithi088.front.sepia.ceph.com /etc/hosts
2024-03-31T16:06:06.132 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88 smithi088.front.sepia.ceph.com smithi088
2024-03-31T16:06:06.133 DEBUG:teuthology.orchestra.run.smithi088:> sudo hostname smithi088
2024-03-31T16:06:06.239 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/smithi088.front.sepia.ceph.com/smithi088/g' /etc/hosts
2024-03-31T16:06:06.267 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/172.21.15.88/172.21.15.88/g' /etc/hosts
2024-03-31T16:06:06.335 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/smithi088.front.sepia.ceph.com/smithi088/g' /etc/hostname
2024-03-31T16:06:06.402 DEBUG:teuthology.orchestra.run.smithi088:> sudo hostname smithi088
2024-03-31T16:06:06.468 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release
2024-03-31T16:06:06.522 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Stream"
2024-03-31T16:06:06.522 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="9"
2024-03-31T16:06:06.522 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos"
2024-03-31T16:06:06.522 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora"
2024-03-31T16:06:06.522 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="9"
2024-03-31T16:06:06.522 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el9"
2024-03-31T16:06:06.522 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-31T16:06:06.522 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31"
2024-03-31T16:06:06.523 INFO:teuthology.orchestra.run.smithi088.stdout:LOGO="fedora-logo-icon"
2024-03-31T16:06:06.523 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-31T16:06:06.523 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://centos.org/"
2024-03-31T16:06:06.523 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-31T16:06:06.523 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-31T16:06:06.523 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-31T16:06:06.523 INFO:teuthology.provision.fog.smithi088:Deploy complete!
2024-03-31T16:06:06.524 DEBUG:teuthology.parallel:result is None
2024-03-31T16:06:06.736 INFO:teuthology.lock.ops:Checking smithi088.front.sepia.ceph.com
2024-03-31T16:06:06.737 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-31T16:06:06.766 INFO:teuthology.lock.ops:Checking smithi099.front.sepia.ceph.com
2024-03-31T16:06:06.766 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-31T16:06:06.790 INFO:teuthology.lock.ops:Checking smithi190.front.sepia.ceph.com
2024-03-31T16:06:06.790 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-31T16:06:06.809 INFO:teuthology.lock.ops:Updating [smithi088.front.sepia.ceph.com]: set os type and version on server
2024-03-31T16:06:06.809 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com'
2024-03-31T16:06:06.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:06:06.894 DEBUG:teuthology.orchestra.run.smithi088:> true
2024-03-31T16:06:06.960 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com'
2024-03-31T16:06:06.961 DEBUG:teuthology.orchestra.run.smithi088:> uname -m
2024-03-31T16:06:07.015 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64
2024-03-31T16:06:07.015 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release
2024-03-31T16:06:07.069 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Stream"
2024-03-31T16:06:07.069 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="9"
2024-03-31T16:06:07.069 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos"
2024-03-31T16:06:07.069 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora"
2024-03-31T16:06:07.069 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="9"
2024-03-31T16:06:07.069 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el9"
2024-03-31T16:06:07.069 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-31T16:06:07.069 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31"
2024-03-31T16:06:07.069 INFO:teuthology.orchestra.run.smithi088.stdout:LOGO="fedora-logo-icon"
2024-03-31T16:06:07.069 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-31T16:06:07.069 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://centos.org/"
2024-03-31T16:06:07.070 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-31T16:06:07.070 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-31T16:06:07.070 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-31T16:06:07.070 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server
2024-03-31T16:06:07.091 INFO:teuthology.lock.ops:Updating [smithi099.front.sepia.ceph.com]: set os type and version on server
2024-03-31T16:06:07.091 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com'
2024-03-31T16:06:07.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:06:07.177 DEBUG:teuthology.orchestra.run.smithi099:> true
2024-03-31T16:06:07.345 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com'
2024-03-31T16:06:07.345 DEBUG:teuthology.orchestra.run.smithi099:> uname -m
2024-03-31T16:06:07.400 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64
2024-03-31T16:06:07.400 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release
2024-03-31T16:06:07.455 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream"
2024-03-31T16:06:07.455 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9"
2024-03-31T16:06:07.455 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos"
2024-03-31T16:06:07.455 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora"
2024-03-31T16:06:07.455 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9"
2024-03-31T16:06:07.455 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9"
2024-03-31T16:06:07.455 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-31T16:06:07.455 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31"
2024-03-31T16:06:07.455 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon"
2024-03-31T16:06:07.455 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-31T16:06:07.455 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/"
2024-03-31T16:06:07.456 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-31T16:06:07.456 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-31T16:06:07.456 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-31T16:06:07.456 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server
2024-03-31T16:06:07.474 INFO:teuthology.lock.ops:Updating [smithi190.front.sepia.ceph.com]: set os type and version on server
2024-03-31T16:06:07.474 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com'
2024-03-31T16:06:07.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-31T16:06:07.561 DEBUG:teuthology.orchestra.run.smithi190:> true
2024-03-31T16:06:07.728 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com'
2024-03-31T16:06:07.728 DEBUG:teuthology.orchestra.run.smithi190:> uname -m
2024-03-31T16:06:07.782 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64
2024-03-31T16:06:07.782 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release
2024-03-31T16:06:07.836 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream"
2024-03-31T16:06:07.836 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="9"
2024-03-31T16:06:07.836 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos"
2024-03-31T16:06:07.836 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora"
2024-03-31T16:06:07.836 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="9"
2024-03-31T16:06:07.836 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el9"
2024-03-31T16:06:07.836 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-31T16:06:07.836 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31"
2024-03-31T16:06:07.836 INFO:teuthology.orchestra.run.smithi190.stdout:LOGO="fedora-logo-icon"
2024-03-31T16:06:07.837 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-31T16:06:07.837 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/"
2024-03-31T16:06:07.837 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-31T16:06:07.837 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-31T16:06:07.837 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-31T16:06:07.837 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server
2024-03-31T16:06:07.857 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-31T16:06:07.956 INFO:teuthology.dispatcher.supervisor:Running job 7631698
2024-03-31T16:06:07.956 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-03-30_23:12:08-fs-wip-vshankar-testing-20240330.170739-testing-default-smithi/7631698 --name vshankar-2024-03-30_23:12:08-fs-wip-vshankar-testing-20240330.170739-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/postgres}} -- /home/teuthworker/archive/vshankar-2024-03-30_23:12:08-fs-wip-vshankar-testing-20240330.170739-testing-default-smithi/7631698/orig.config.yaml
2024-03-31T16:06:07.960 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-03-30_23:12:08-fs-wip-vshankar-testing-20240330.170739-testing-default-smithi/7631698
2024-03-31T16:06:07.960 INFO:teuthology.dispatcher.supervisor:Job PID: 220848
2024-03-31T16:06:07.960 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-03-31T16:08:08.059 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-31T16:10:08.256 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-31T16:12:08.423 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-31T16:14:08.599 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-31T16:16:08.766 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-31T16:18:08.931 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-31T16:18:08.996 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1