2024-03-18T15:17:51.482 INFO:root:teuthology version: 0.0.1.dev262+ge691533
2024-03-18T15:17:51.482 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:17:51.662 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fd616162f70>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fd6171c3c10>, signals=[15])
2024-03-18T15:17:51.663 INFO:teuthology.lock.ops:Start node 'smithi027.front.sepia.ceph.com' reimaging
2024-03-18T15:17:51.663 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: reset os type and version on server
2024-03-18T15:17:51.663 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server
2024-03-18T15:17:51.686 INFO:teuthology.lock.ops:Start node 'smithi077.front.sepia.ceph.com' reimaging
2024-03-18T15:17:51.686 INFO:teuthology.lock.ops:Updating [smithi077.front.sepia.ceph.com]: reset os type and version on server
2024-03-18T15:17:51.686 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server
2024-03-18T15:17:51.702 INFO:teuthology.lock.ops:Start node 'smithi122.front.sepia.ceph.com' reimaging
2024-03-18T15:17:51.702 INFO:teuthology.lock.ops:Updating [smithi122.front.sepia.ceph.com]: reset os type and version on server
2024-03-18T15:17:51.702 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server
2024-03-18T15:17:51.799 DEBUG:teuthology.provision.fog.smithi077:Requesting image smithi_centos_9.stream (ID 10)
2024-03-18T15:17:51.802 DEBUG:teuthology.provision.fog.smithi027:Requesting image smithi_centos_9.stream (ID 10)
2024-03-18T15:17:51.812 DEBUG:teuthology.provision.fog.smithi122:Requesting image smithi_centos_9.stream (ID 10)
2024-03-18T15:17:51.838 INFO:teuthology.provision.fog.smithi077:Scheduling deploy of centos 9.stream
2024-03-18T15:17:51.843 INFO:teuthology.provision.fog.smithi027:Scheduling deploy of centos 9.stream
2024-03-18T15:17:51.857 INFO:teuthology.provision.fog.smithi122:Scheduling deploy of centos 9.stream
2024-03-18T15:17:52.080 INFO:teuthology.orchestra.console.smithi027:Power off
2024-03-18T15:17:52.080 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-18T15:17:52.097 INFO:teuthology.orchestra.console.smithi122:Power off
2024-03-18T15:17:52.097 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-18T15:17:52.109 INFO:teuthology.orchestra.console.smithi077:Power off
2024-03-18T15:17:52.109 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-18T15:17:52.117 DEBUG:teuthology.orchestra.console.smithi027:power off output: Chassis Power Control: Down/Off
2024-03-18T15:17:52.118 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T15:17:52.126 DEBUG:teuthology.orchestra.console.smithi122:power off output: Chassis Power Control: Down/Off
2024-03-18T15:17:52.126 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T15:17:52.136 DEBUG:teuthology.orchestra.console.smithi077:power off output: Chassis Power Control: Down/Off
2024-03-18T15:17:52.137 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T15:17:52.146 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off
2024-03-18T15:17:52.149 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off
2024-03-18T15:17:52.163 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is off
2024-03-18T15:17:52.248 INFO:teuthology.orchestra.console.smithi027:Power off completed
2024-03-18T15:17:52.249 INFO:teuthology.orchestra.console.smithi122:Power off completed
2024-03-18T15:17:52.265 INFO:teuthology.orchestra.console.smithi077:Power off completed
2024-03-18T15:17:52.349 INFO:teuthology.orchestra.console.smithi027:Power on
2024-03-18T15:17:52.349 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-18T15:17:52.357 INFO:teuthology.orchestra.console.smithi122:Power on
2024-03-18T15:17:52.358 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-18T15:17:52.367 INFO:teuthology.orchestra.console.smithi077:Power on
2024-03-18T15:17:52.367 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-18T15:17:52.375 DEBUG:teuthology.orchestra.console.smithi027:power on output: Chassis Power Control: Up/On
2024-03-18T15:17:52.375 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T15:17:52.384 DEBUG:teuthology.orchestra.console.smithi122:power on output: Chassis Power Control: Up/On
2024-03-18T15:17:52.384 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T15:17:52.392 DEBUG:teuthology.orchestra.console.smithi077:power on output: Chassis Power Control: Up/On
2024-03-18T15:17:52.393 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T15:17:52.401 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off
2024-03-18T15:17:52.405 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off
2024-03-18T15:17:52.412 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is off
2024-03-18T15:17:56.404 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T15:17:56.412 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T15:17:56.419 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T15:17:56.515 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on
2024-03-18T15:17:56.521 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on
2024-03-18T15:17:56.526 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is off
2024-03-18T15:17:56.616 INFO:teuthology.orchestra.console.smithi027:Power on completed
2024-03-18T15:17:56.621 INFO:teuthology.orchestra.console.smithi122:Power on completed
2024-03-18T15:17:56.717 INFO:teuthology.provision.fog.smithi027:Waiting for deploy to finish
2024-03-18T15:17:56.722 INFO:teuthology.provision.fog.smithi122:Waiting for deploy to finish
2024-03-18T15:18:00.531 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T15:18:00.642 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is on
2024-03-18T15:18:00.742 INFO:teuthology.orchestra.console.smithi077:Power on completed
2024-03-18T15:18:00.843 INFO:teuthology.provision.fog.smithi077:Waiting for deploy to finish
2024-03-18T15:20:28.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:20:28.223 ERROR:teuthology.orchestra.connection:Error authenticating with smithi122.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-18T15:20:47.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:20:50.306 WARNING:teuthology.provision.fog.smithi077:[Errno None] Unable to connect to port 22 on 172.21.15.77
2024-03-18T15:20:56.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:20:59.358 WARNING:teuthology.provision.fog.smithi077:[Errno None] Unable to connect to port 22 on 172.21.15.77
2024-03-18T15:21:05.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:21:08.414 WARNING:teuthology.provision.fog.smithi077:[Errno None] Unable to connect to port 22 on 172.21.15.77
2024-03-18T15:21:14.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:21:17.470 WARNING:teuthology.provision.fog.smithi077:[Errno None] Unable to connect to port 22 on 172.21.15.77
2024-03-18T15:21:23.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:21:26.530 WARNING:teuthology.provision.fog.smithi077:[Errno None] Unable to connect to port 22 on 172.21.15.77
2024-03-18T15:21:28.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:21:31.486 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-03-18T15:21:32.226 WARNING:teuthology.provision.fog.smithi122:timed out
2024-03-18T15:21:32.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:21:35.586 WARNING:teuthology.provision.fog.smithi077:[Errno None] Unable to connect to port 22 on 172.21.15.77
2024-03-18T15:21:37.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:21:38.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:21:40.546 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-03-18T15:21:41.278 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122
2024-03-18T15:21:41.589 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:21:46.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:21:47.280 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:21:49.598 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-03-18T15:21:50.334 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122
2024-03-18T15:21:55.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:21:56.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:21:58.658 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-03-18T15:21:59.394 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122
2024-03-18T15:22:04.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:22:05.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:22:07.714 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-03-18T15:22:08.273 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-18T15:22:08.274 DEBUG:teuthology.orchestra.run.smithi122:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-18T15:22:13.716 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:22:16.766 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-03-18T15:22:22.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:22:25.826 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-03-18T15:22:28.714 INFO:teuthology.provision.fog.smithi122:Node is ready
2024-03-18T15:22:28.715 DEBUG:teuthology.orchestra.run.smithi122:> hostname
2024-03-18T15:22:28.732 INFO:teuthology.orchestra.run.smithi122.stdout:smithi046
2024-03-18T15:22:28.733 DEBUG:teuthology.orchestra.run.smithi122:> grep smithi046 /etc/hosts
2024-03-18T15:22:28.790 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046
2024-03-18T15:22:28.790 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname smithi122
2024-03-18T15:22:28.959 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/smithi046/smithi122/g' /etc/hosts
2024-03-18T15:22:28.992 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/172.21.15.46/172.21.15.122/g' /etc/hosts
2024-03-18T15:22:29.067 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/smithi046/smithi122/g' /etc/hostname
2024-03-18T15:22:29.146 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname smithi122
2024-03-18T15:22:29.214 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release
2024-03-18T15:22:29.268 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream"
2024-03-18T15:22:29.268 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9"
2024-03-18T15:22:29.269 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos"
2024-03-18T15:22:29.269 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora"
2024-03-18T15:22:29.269 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9"
2024-03-18T15:22:29.269 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9"
2024-03-18T15:22:29.269 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-18T15:22:29.269 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31"
2024-03-18T15:22:29.269 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon"
2024-03-18T15:22:29.269 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-18T15:22:29.269 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/"
2024-03-18T15:22:29.269 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-18T15:22:29.269 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-18T15:22:29.269 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-18T15:22:29.270 INFO:teuthology.provision.fog.smithi122:Deploy complete!
2024-03-18T15:22:29.272 DEBUG:teuthology.parallel:result is None
2024-03-18T15:22:31.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:22:34.878 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-03-18T15:22:40.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:22:41.591 WARNING:teuthology.provision.fog.smithi077:timed out
2024-03-18T15:22:43.938 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27
2024-03-18T15:22:47.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:22:48.109 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-18T15:22:48.109 DEBUG:teuthology.orchestra.run.smithi077:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-18T15:22:48.436 INFO:teuthology.provision.fog.smithi077:Node is ready
2024-03-18T15:22:48.437 DEBUG:teuthology.orchestra.run.smithi077:> hostname
2024-03-18T15:22:48.451 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077.front.sepia.ceph.com
2024-03-18T15:22:48.452 DEBUG:teuthology.orchestra.run.smithi077:> grep smithi077.front.sepia.ceph.com /etc/hosts
2024-03-18T15:22:48.507 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77 smithi077.front.sepia.ceph.com smithi077
2024-03-18T15:22:48.508 DEBUG:teuthology.orchestra.run.smithi077:> sudo hostname smithi077
2024-03-18T15:22:48.608 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i -e 's/smithi077.front.sepia.ceph.com/smithi077/g' /etc/hosts
2024-03-18T15:22:48.635 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i -e 's/172.21.15.77/172.21.15.77/g' /etc/hosts
2024-03-18T15:22:48.702 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i -e 's/smithi077.front.sepia.ceph.com/smithi077/g' /etc/hostname
2024-03-18T15:22:48.770 DEBUG:teuthology.orchestra.run.smithi077:> sudo hostname smithi077
2024-03-18T15:22:48.837 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release
2024-03-18T15:22:48.892 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream"
2024-03-18T15:22:48.892 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="9"
2024-03-18T15:22:48.892 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos"
2024-03-18T15:22:48.892 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora"
2024-03-18T15:22:48.892 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="9"
2024-03-18T15:22:48.892 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el9"
2024-03-18T15:22:48.892 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-18T15:22:48.892 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31"
2024-03-18T15:22:48.892 INFO:teuthology.orchestra.run.smithi077.stdout:LOGO="fedora-logo-icon"
2024-03-18T15:22:48.892 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-18T15:22:48.892 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/"
2024-03-18T15:22:48.892 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-18T15:22:48.893 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-18T15:22:48.893 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-18T15:22:48.893 INFO:teuthology.provision.fog.smithi077:Deploy complete!
2024-03-18T15:22:48.894 DEBUG:teuthology.parallel:result is None
2024-03-18T15:22:49.940 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:23:49.943 WARNING:teuthology.provision.fog.smithi027:timed out
2024-03-18T15:23:55.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:23:56.216 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-18T15:23:56.217 DEBUG:teuthology.orchestra.run.smithi027:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-18T15:23:56.544 INFO:teuthology.provision.fog.smithi027:Node is ready
2024-03-18T15:23:56.544 DEBUG:teuthology.orchestra.run.smithi027:> hostname
2024-03-18T15:23:56.559 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027.front.sepia.ceph.com
2024-03-18T15:23:56.559 DEBUG:teuthology.orchestra.run.smithi027:> grep smithi027.front.sepia.ceph.com /etc/hosts
2024-03-18T15:23:56.614 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27 smithi027.front.sepia.ceph.com smithi027
2024-03-18T15:23:56.614 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname smithi027
2024-03-18T15:23:56.733 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/smithi027.front.sepia.ceph.com/smithi027/g' /etc/hosts
2024-03-18T15:23:56.765 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/172.21.15.27/172.21.15.27/g' /etc/hosts
2024-03-18T15:23:56.835 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/smithi027.front.sepia.ceph.com/smithi027/g' /etc/hostname
2024-03-18T15:23:56.907 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname smithi027
2024-03-18T15:23:56.977 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release
2024-03-18T15:23:57.032 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream"
2024-03-18T15:23:57.033 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9"
2024-03-18T15:23:57.033 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos"
2024-03-18T15:23:57.033 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora"
2024-03-18T15:23:57.033 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9"
2024-03-18T15:23:57.033 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9"
2024-03-18T15:23:57.033 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-18T15:23:57.033 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31"
2024-03-18T15:23:57.033 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon"
2024-03-18T15:23:57.033 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-18T15:23:57.033 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/"
2024-03-18T15:23:57.033 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-18T15:23:57.033 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-18T15:23:57.033 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-18T15:23:57.034 INFO:teuthology.provision.fog.smithi027:Deploy complete!
2024-03-18T15:23:57.035 DEBUG:teuthology.parallel:result is None
2024-03-18T15:23:57.284 INFO:teuthology.lock.ops:Checking smithi027.front.sepia.ceph.com
2024-03-18T15:23:57.284 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-18T15:23:57.304 INFO:teuthology.lock.ops:Checking smithi077.front.sepia.ceph.com
2024-03-18T15:23:57.304 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-18T15:23:57.328 INFO:teuthology.lock.ops:Checking smithi122.front.sepia.ceph.com
2024-03-18T15:23:57.328 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-18T15:23:57.346 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: set os type and version on server
2024-03-18T15:23:57.346 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com'
2024-03-18T15:23:57.347 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:23:57.487 DEBUG:teuthology.orchestra.run.smithi027:> true
2024-03-18T15:23:57.552 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com'
2024-03-18T15:23:57.552 DEBUG:teuthology.orchestra.run.smithi027:> uname -m
2024-03-18T15:23:57.607 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64
2024-03-18T15:23:57.608 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release
2024-03-18T15:23:57.662 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream"
2024-03-18T15:23:57.662 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9"
2024-03-18T15:23:57.662 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos"
2024-03-18T15:23:57.663 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora"
2024-03-18T15:23:57.663 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9"
2024-03-18T15:23:57.663 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9"
2024-03-18T15:23:57.663 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-18T15:23:57.663 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31"
2024-03-18T15:23:57.663 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon"
2024-03-18T15:23:57.663 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-18T15:23:57.663 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/"
2024-03-18T15:23:57.663 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-18T15:23:57.663 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-18T15:23:57.663 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-18T15:23:57.664 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server
2024-03-18T15:23:57.704 INFO:teuthology.lock.ops:Updating [smithi077.front.sepia.ceph.com]: set os type and version on server
2024-03-18T15:23:57.704 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com'
2024-03-18T15:23:57.704 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:23:57.792 DEBUG:teuthology.orchestra.run.smithi077:> true
2024-03-18T15:23:57.956 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com'
2024-03-18T15:23:57.956 DEBUG:teuthology.orchestra.run.smithi077:> uname -m
2024-03-18T15:23:58.011 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64
2024-03-18T15:23:58.011 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release
2024-03-18T15:23:58.066 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream"
2024-03-18T15:23:58.066 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="9"
2024-03-18T15:23:58.066 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos"
2024-03-18T15:23:58.066 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora"
2024-03-18T15:23:58.066 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="9"
2024-03-18T15:23:58.066 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el9"
2024-03-18T15:23:58.066 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-18T15:23:58.066 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31"
2024-03-18T15:23:58.067 INFO:teuthology.orchestra.run.smithi077.stdout:LOGO="fedora-logo-icon"
2024-03-18T15:23:58.067 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-18T15:23:58.067 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/"
2024-03-18T15:23:58.067 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-18T15:23:58.067 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-18T15:23:58.067 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-18T15:23:58.067 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server
2024-03-18T15:23:58.104 INFO:teuthology.lock.ops:Updating [smithi122.front.sepia.ceph.com]: set os type and version on server
2024-03-18T15:23:58.104 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com'
2024-03-18T15:23:58.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T15:23:58.193 DEBUG:teuthology.orchestra.run.smithi122:> true
2024-03-18T15:23:58.371 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com'
2024-03-18T15:23:58.371 DEBUG:teuthology.orchestra.run.smithi122:> uname -m
2024-03-18T15:23:58.427 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64
2024-03-18T15:23:58.428 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release
2024-03-18T15:23:58.483 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream"
2024-03-18T15:23:58.483 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9"
2024-03-18T15:23:58.483 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos"
2024-03-18T15:23:58.483 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora"
2024-03-18T15:23:58.483 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9"
2024-03-18T15:23:58.483 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9"
2024-03-18T15:23:58.483 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-18T15:23:58.483 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31"
2024-03-18T15:23:58.483 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon"
2024-03-18T15:23:58.484 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-18T15:23:58.484 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/"
2024-03-18T15:23:58.484 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-18T15:23:58.484 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-18T15:23:58.484 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-18T15:23:58.484 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server
2024-03-18T15:23:58.504 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:23:58.600 INFO:teuthology.dispatcher.supervisor:Running job 7608884
2024-03-18T15:23:58.601 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608884 --name pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} -- /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608884/orig.config.yaml
2024-03-18T15:23:58.603 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608884
2024-03-18T15:23:58.604 INFO:teuthology.dispatcher.supervisor:Job PID: 945675
2024-03-18T15:23:58.604 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-03-18T15:25:58.606 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:27:58.682 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:29:58.762 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:31:58.836 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:33:58.909 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:35:58.973 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:37:59.041 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:39:59.123 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:41:59.200 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:43:59.271 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:45:59.355 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:47:59.433 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:49:59.497 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:51:59.595 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:53:59.665 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:55:59.739 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:57:59.805 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T15:59:59.879 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T16:01:59.953 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T16:04:00.023 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T16:06:00.098 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T16:06:00.152 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1