2024-03-23T23:12:55.270 INFO:root:teuthology version: 0.0.1.dev262+ge691533
2024-03-23T23:12:55.270 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:12:55.426 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fd41c277f40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fd41d2d8c10>, signals=[15])
2024-03-23T23:12:55.427 INFO:teuthology.lock.ops:Start node 'smithi099.front.sepia.ceph.com' reimaging
2024-03-23T23:12:55.427 INFO:teuthology.lock.ops:Updating [smithi099.front.sepia.ceph.com]: reset os type and version on server
2024-03-23T23:12:55.427 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server
2024-03-23T23:12:55.449 INFO:teuthology.lock.ops:Start node 'smithi149.front.sepia.ceph.com' reimaging
2024-03-23T23:12:55.450 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: reset os type and version on server
2024-03-23T23:12:55.450 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server
2024-03-23T23:12:55.466 INFO:teuthology.lock.ops:Start node 'smithi174.front.sepia.ceph.com' reimaging
2024-03-23T23:12:55.466 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: reset os type and version on server
2024-03-23T23:12:55.466 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server
2024-03-23T23:12:55.663 DEBUG:teuthology.provision.fog.smithi149:Requesting image smithi_centos_9.stream (ID 10)
2024-03-23T23:12:55.667 DEBUG:teuthology.provision.fog.smithi174:Requesting image smithi_centos_9.stream (ID 10)
2024-03-23T23:12:55.676 DEBUG:teuthology.provision.fog.smithi099:Requesting image smithi_centos_9.stream (ID 10)
2024-03-23T23:12:55.710 INFO:teuthology.provision.fog.smithi174:Scheduling deploy of centos 9.stream
2024-03-23T23:12:55.717 INFO:teuthology.provision.fog.smithi149:Scheduling deploy of centos 9.stream
2024-03-23T23:12:55.731 INFO:teuthology.provision.fog.smithi099:Scheduling deploy of centos 9.stream
2024-03-23T23:12:55.930 INFO:teuthology.orchestra.console.smithi174:Power off
2024-03-23T23:12:55.931 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-23T23:12:55.945 INFO:teuthology.orchestra.console.smithi099:Power off
2024-03-23T23:12:55.946 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-23T23:12:55.957 DEBUG:teuthology.orchestra.console.smithi174:power off output: Chassis Power Control: Down/Off
2024-03-23T23:12:55.957 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-23T23:12:55.967 DEBUG:teuthology.orchestra.console.smithi099:power off output: Chassis Power Control: Down/Off
2024-03-23T23:12:55.968 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-23T23:12:55.979 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on
2024-03-23T23:12:55.989 INFO:teuthology.orchestra.console.smithi149:Power off
2024-03-23T23:12:55.989 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-23T23:12:55.996 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on
2024-03-23T23:12:56.009 DEBUG:teuthology.orchestra.console.smithi149:power off output: Chassis Power Control: Down/Off
2024-03-23T23:12:56.010 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-23T23:12:56.029 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on
2024-03-23T23:12:59.985 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-23T23:12:59.998 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-23T23:13:00.031 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-23T23:13:00.094 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on
2024-03-23T23:13:00.106 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on
2024-03-23T23:13:00.139 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on
2024-03-23T23:13:04.099 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-23T23:13:04.107 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-23T23:13:04.141 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-23T23:13:04.208 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off
2024-03-23T23:13:04.216 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off
2024-03-23T23:13:04.248 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off
2024-03-23T23:13:04.309 INFO:teuthology.orchestra.console.smithi174:Power off completed
2024-03-23T23:13:04.318 INFO:teuthology.orchestra.console.smithi099:Power off completed
2024-03-23T23:13:04.349 INFO:teuthology.orchestra.console.smithi149:Power off completed
2024-03-23T23:13:04.411 INFO:teuthology.orchestra.console.smithi174:Power on
2024-03-23T23:13:04.411 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-23T23:13:04.419 INFO:teuthology.orchestra.console.smithi099:Power on
2024-03-23T23:13:04.419 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-23T23:13:04.440 DEBUG:teuthology.orchestra.console.smithi099:power on output: Chassis Power Control: Up/On
2024-03-23T23:13:04.440 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-23T23:13:04.448 DEBUG:teuthology.orchestra.console.smithi174:power on output: Chassis Power Control: Up/On
2024-03-23T23:13:04.449 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-23T23:13:04.455 INFO:teuthology.orchestra.console.smithi149:Power on
2024-03-23T23:13:04.455 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-23T23:13:04.464 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off
2024-03-23T23:13:04.470 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off
2024-03-23T23:13:04.476 DEBUG:teuthology.orchestra.console.smithi149:power on output: Chassis Power Control: Up/On
2024-03-23T23:13:04.477 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-23T23:13:04.496 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off
2024-03-23T23:13:08.468 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-23T23:13:08.475 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-23T23:13:08.497 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-23T23:13:08.578 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off
2024-03-23T23:13:08.583 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off
2024-03-23T23:13:08.606 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on
2024-03-23T23:13:08.706 INFO:teuthology.orchestra.console.smithi149:Power on completed
2024-03-23T23:13:08.807 INFO:teuthology.provision.fog.smithi149:Waiting for deploy to finish
2024-03-23T23:13:12.582 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-23T23:13:12.589 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-23T23:13:12.691 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on
2024-03-23T23:13:12.698 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on
2024-03-23T23:13:12.792 INFO:teuthology.orchestra.console.smithi099:Power on completed
2024-03-23T23:13:12.799 INFO:teuthology.orchestra.console.smithi174:Power on completed
2024-03-23T23:13:12.893 INFO:teuthology.provision.fog.smithi099:Waiting for deploy to finish
2024-03-23T23:13:12.899 INFO:teuthology.provision.fog.smithi174:Waiting for deploy to finish
2024-03-23T23:15:43.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:15:46.622 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174
2024-03-23T23:15:52.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:15:54.566 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:15:55.678 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174
2024-03-23T23:15:57.634 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2024-03-23T23:16:01.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:16:03.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:16:04.738 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174
2024-03-23T23:16:06.722 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2024-03-23T23:16:10.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:16:12.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:16:13.822 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174
2024-03-23T23:16:15.806 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2024-03-23T23:16:19.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:16:21.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:16:22.878 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174
2024-03-23T23:16:24.862 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2024-03-23T23:16:28.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:16:30.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:16:31.934 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174
2024-03-23T23:16:33.918 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2024-03-23T23:16:37.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:16:39.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:16:40.990 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174
2024-03-23T23:16:42.974 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2024-03-23T23:16:46.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:16:48.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:16:52.030 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2024-03-23T23:16:58.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:16:58.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:17:01.918 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-03-23T23:17:07.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:17:10.978 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-03-23T23:17:16.450 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2024-03-23T23:17:16.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:17:20.030 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-03-23T23:17:22.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:17:25.534 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2024-03-23T23:17:26.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:17:29.086 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-03-23T23:17:31.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:17:34.818 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-23T23:17:34.819 DEBUG:teuthology.orchestra.run.smithi149:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-23T23:17:35.088 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:17:38.142 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-03-23T23:17:44.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:17:46.997 WARNING:teuthology.provision.fog.smithi174:timed out
2024-03-23T23:17:47.198 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2024-03-23T23:17:52.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:17:53.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:17:53.256 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-23T23:17:53.256 DEBUG:teuthology.orchestra.run.smithi174:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-23T23:17:53.594 INFO:teuthology.provision.fog.smithi174:Node is ready
2024-03-23T23:17:53.594 DEBUG:teuthology.orchestra.run.smithi174:> hostname
2024-03-23T23:17:53.608 INFO:teuthology.orchestra.run.smithi174.stdout:smithi174.front.sepia.ceph.com
2024-03-23T23:17:53.608 DEBUG:teuthology.orchestra.run.smithi174:> grep smithi174.front.sepia.ceph.com /etc/hosts
2024-03-23T23:17:53.666 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.15.174 smithi174.front.sepia.ceph.com smithi174
2024-03-23T23:17:53.667 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname smithi174
2024-03-23T23:17:53.786 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/smithi174.front.sepia.ceph.com/smithi174/g' /etc/hosts
2024-03-23T23:17:53.813 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/172.21.15.174/172.21.15.174/g' /etc/hosts
2024-03-23T23:17:53.884 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/smithi174.front.sepia.ceph.com/smithi174/g' /etc/hostname
2024-03-23T23:17:53.956 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname smithi174
2024-03-23T23:17:54.026 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release
2024-03-23T23:17:54.083 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream"
2024-03-23T23:17:54.084 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9"
2024-03-23T23:17:54.084 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos"
2024-03-23T23:17:54.084 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora"
2024-03-23T23:17:54.084 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9"
2024-03-23T23:17:54.084 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9"
2024-03-23T23:17:54.084 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-23T23:17:54.084 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31"
2024-03-23T23:17:54.084 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon"
2024-03-23T23:17:54.084 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-23T23:17:54.084 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/"
2024-03-23T23:17:54.085 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-23T23:17:54.085 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-23T23:17:54.085 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-23T23:17:54.085 INFO:teuthology.provision.fog.smithi174:Deploy complete!
2024-03-23T23:17:54.087 DEBUG:teuthology.parallel:result is None
2024-03-23T23:18:00.718 INFO:teuthology.provision.fog.smithi149:Node is ready
2024-03-23T23:18:00.718 DEBUG:teuthology.orchestra.run.smithi149:> hostname
2024-03-23T23:18:00.737 INFO:teuthology.orchestra.run.smithi149.stdout:smithi149.front.sepia.ceph.com
2024-03-23T23:18:00.737 DEBUG:teuthology.orchestra.run.smithi149:> grep smithi149.front.sepia.ceph.com /etc/hosts
2024-03-23T23:18:00.795 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.15.149 smithi149.front.sepia.ceph.com smithi149
2024-03-23T23:18:00.795 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname smithi149
2024-03-23T23:18:00.938 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/smithi149.front.sepia.ceph.com/smithi149/g' /etc/hosts
2024-03-23T23:18:00.969 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/172.21.15.149/172.21.15.149/g' /etc/hosts
2024-03-23T23:18:01.040 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/smithi149.front.sepia.ceph.com/smithi149/g' /etc/hostname
2024-03-23T23:18:01.110 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname smithi149
2024-03-23T23:18:01.181 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release
2024-03-23T23:18:01.237 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream"
2024-03-23T23:18:01.237 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9"
2024-03-23T23:18:01.237 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos"
2024-03-23T23:18:01.238 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora"
2024-03-23T23:18:01.238 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9"
2024-03-23T23:18:01.238 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9"
2024-03-23T23:18:01.238 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-23T23:18:01.238 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31"
2024-03-23T23:18:01.238 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon"
2024-03-23T23:18:01.238 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-23T23:18:01.238 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/"
2024-03-23T23:18:01.238 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-23T23:18:01.238 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-23T23:18:01.238 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-23T23:18:01.239 INFO:teuthology.provision.fog.smithi149:Deploy complete!
2024-03-23T23:18:01.239 DEBUG:teuthology.parallel:result is None
2024-03-23T23:18:53.251 WARNING:teuthology.provision.fog.smithi099:timed out
2024-03-23T23:18:59.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:18:59.534 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-23T23:18:59.534 DEBUG:teuthology.orchestra.run.smithi099:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-23T23:18:59.870 INFO:teuthology.provision.fog.smithi099:Node is ready
2024-03-23T23:18:59.871 DEBUG:teuthology.orchestra.run.smithi099:> hostname
2024-03-23T23:18:59.886 INFO:teuthology.orchestra.run.smithi099.stdout:smithi099.front.sepia.ceph.com
2024-03-23T23:18:59.886 DEBUG:teuthology.orchestra.run.smithi099:> grep smithi099.front.sepia.ceph.com /etc/hosts
2024-03-23T23:18:59.942 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99 smithi099.front.sepia.ceph.com smithi099
2024-03-23T23:18:59.942 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname smithi099
2024-03-23T23:19:00.071 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/smithi099.front.sepia.ceph.com/smithi099/g' /etc/hosts
2024-03-23T23:19:00.100 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/172.21.15.99/172.21.15.99/g' /etc/hosts
2024-03-23T23:19:00.170 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/smithi099.front.sepia.ceph.com/smithi099/g' /etc/hostname
2024-03-23T23:19:00.239 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname smithi099
2024-03-23T23:19:00.306 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release
2024-03-23T23:19:00.360 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream"
2024-03-23T23:19:00.360 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9"
2024-03-23T23:19:00.361 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos"
2024-03-23T23:19:00.361 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora"
2024-03-23T23:19:00.361 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9"
2024-03-23T23:19:00.361 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9"
2024-03-23T23:19:00.361 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-23T23:19:00.361 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31"
2024-03-23T23:19:00.361 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon"
2024-03-23T23:19:00.361 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-23T23:19:00.361 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/"
2024-03-23T23:19:00.361 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-23T23:19:00.361 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-23T23:19:00.361 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-23T23:19:00.362 INFO:teuthology.provision.fog.smithi099:Deploy complete!
2024-03-23T23:19:00.362 DEBUG:teuthology.parallel:result is None
2024-03-23T23:19:00.578 INFO:teuthology.lock.ops:Checking smithi099.front.sepia.ceph.com
2024-03-23T23:19:00.578 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-23T23:19:00.633 INFO:teuthology.lock.ops:Checking smithi149.front.sepia.ceph.com
2024-03-23T23:19:00.634 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-23T23:19:00.664 INFO:teuthology.lock.ops:Checking smithi174.front.sepia.ceph.com
2024-03-23T23:19:00.664 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-23T23:19:00.682 INFO:teuthology.lock.ops:Updating [smithi099.front.sepia.ceph.com]: set os type and version on server
2024-03-23T23:19:00.682 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com'
2024-03-23T23:19:00.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:19:00.773 DEBUG:teuthology.orchestra.run.smithi099:> true
2024-03-23T23:19:00.844 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com'
2024-03-23T23:19:00.844 DEBUG:teuthology.orchestra.run.smithi099:> uname -m
2024-03-23T23:19:00.898 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64
2024-03-23T23:19:00.898 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release
2024-03-23T23:19:00.952 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream"
2024-03-23T23:19:00.952 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9"
2024-03-23T23:19:00.952 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos"
2024-03-23T23:19:00.952 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora"
2024-03-23T23:19:00.953 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9"
2024-03-23T23:19:00.953 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9"
2024-03-23T23:19:00.953 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-23T23:19:00.953 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31"
2024-03-23T23:19:00.953 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon"
2024-03-23T23:19:00.953 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-23T23:19:00.953 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/"
2024-03-23T23:19:00.953 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-23T23:19:00.953 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-23T23:19:00.953 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-23T23:19:00.954 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server
2024-03-23T23:19:00.981 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: set os type and version on server
2024-03-23T23:19:00.981 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com'
2024-03-23T23:19:00.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:19:01.071 DEBUG:teuthology.orchestra.run.smithi149:> true
2024-03-23T23:19:01.244 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com'
2024-03-23T23:19:01.245 DEBUG:teuthology.orchestra.run.smithi149:> uname -m
2024-03-23T23:19:01.300 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64
2024-03-23T23:19:01.300 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release
2024-03-23T23:19:01.354 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream"
2024-03-23T23:19:01.355 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9"
2024-03-23T23:19:01.355 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos"
2024-03-23T23:19:01.355 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora"
2024-03-23T23:19:01.355 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9"
2024-03-23T23:19:01.355 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9"
2024-03-23T23:19:01.355 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-23T23:19:01.355 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31"
2024-03-23T23:19:01.355 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon"
2024-03-23T23:19:01.355 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-23T23:19:01.355 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/"
2024-03-23T23:19:01.355 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-23T23:19:01.355 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-23T23:19:01.356 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-23T23:19:01.356 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server
2024-03-23T23:19:01.381 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: set os type and version on server
2024-03-23T23:19:01.381 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com'
2024-03-23T23:19:01.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-23T23:19:01.470 DEBUG:teuthology.orchestra.run.smithi174:> true
2024-03-23T23:19:01.640 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com'
2024-03-23T23:19:01.641 DEBUG:teuthology.orchestra.run.smithi174:> uname -m
2024-03-23T23:19:01.695 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64
2024-03-23T23:19:01.696 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release
2024-03-23T23:19:01.750 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream"
2024-03-23T23:19:01.751 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9"
2024-03-23T23:19:01.751 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos"
2024-03-23T23:19:01.751 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora"
2024-03-23T23:19:01.751 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9"
2024-03-23T23:19:01.751 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9"
2024-03-23T23:19:01.751 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-23T23:19:01.751 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31"
2024-03-23T23:19:01.751 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon"
2024-03-23T23:19:01.751 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-23T23:19:01.751 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/"
2024-03-23T23:19:01.751 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-23T23:19:01.751 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-23T23:19:01.751 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-23T23:19:01.752 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server
2024-03-23T23:19:01.781 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:19:01.876 INFO:teuthology.dispatcher.supervisor:Running job 7616393
2024-03-23T23:19:01.877 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner mchangir --archive /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393 --name mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/direct_io}} -- /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393/orig.config.yaml
2024-03-23T23:19:01.880 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/mchangir-2024-03-22_09:49:57-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7616393
2024-03-23T23:19:01.880 INFO:teuthology.dispatcher.supervisor:Job PID: 2622187
2024-03-23T23:19:01.880 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-03-23T23:21:01.981 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:23:02.136 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:25:02.305 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:27:02.461 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:29:02.631 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:31:02.811 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:33:02.979 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:35:03.118 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:37:03.283 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:39:03.523 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:41:03.695 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:43:03.863 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:45:04.059 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:47:04.236 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:49:04.407 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:51:04.584 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-23T23:51:04.655 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1