2024-03-11T05:14:45.068 INFO:root:teuthology version: 0.0.1.dev262+ge691533
2024-03-11T05:14:45.069 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:14:45.244 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f3e05fe7f40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f3e06f49c10>, signals=[15])
2024-03-11T05:14:45.244 INFO:teuthology.lock.ops:Start node 'smithi040.front.sepia.ceph.com' reimaging
2024-03-11T05:14:45.245 INFO:teuthology.lock.ops:Updating [smithi040.front.sepia.ceph.com]: reset os type and version on server
2024-03-11T05:14:45.245 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server
2024-03-11T05:14:45.263 INFO:teuthology.lock.ops:Start node 'smithi063.front.sepia.ceph.com' reimaging
2024-03-11T05:14:45.263 INFO:teuthology.lock.ops:Updating [smithi063.front.sepia.ceph.com]: reset os type and version on server
2024-03-11T05:14:45.263 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server
2024-03-11T05:14:45.280 INFO:teuthology.lock.ops:Start node 'smithi149.front.sepia.ceph.com' reimaging
2024-03-11T05:14:45.280 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: reset os type and version on server
2024-03-11T05:14:45.280 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server
2024-03-11T05:14:45.383 DEBUG:teuthology.provision.fog.smithi063:Requesting image smithi_centos_9.stream (ID 10)
2024-03-11T05:14:45.385 DEBUG:teuthology.provision.fog.smithi040:Requesting image smithi_centos_9.stream (ID 10)
2024-03-11T05:14:45.399 DEBUG:teuthology.provision.fog.smithi149:Requesting image smithi_centos_9.stream (ID 10)
2024-03-11T05:14:45.429 INFO:teuthology.provision.fog.smithi063:Scheduling deploy of centos 9.stream
2024-03-11T05:14:45.432 INFO:teuthology.provision.fog.smithi040:Scheduling deploy of centos 9.stream
2024-03-11T05:14:45.442 INFO:teuthology.provision.fog.smithi149:Scheduling deploy of centos 9.stream
2024-03-11T05:14:45.642 INFO:teuthology.orchestra.console.smithi063:Power off
2024-03-11T05:14:45.643 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-11T05:14:45.656 INFO:teuthology.orchestra.console.smithi040:Power off
2024-03-11T05:14:45.657 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-11T05:14:45.669 INFO:teuthology.orchestra.console.smithi149:Power off
2024-03-11T05:14:45.670 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-11T05:14:45.679 DEBUG:teuthology.orchestra.console.smithi063:power off output: Chassis Power Control: Down/Off
2024-03-11T05:14:45.679 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:14:45.688 DEBUG:teuthology.orchestra.console.smithi040:power off output: Chassis Power Control: Down/Off
2024-03-11T05:14:45.688 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:14:45.698 DEBUG:teuthology.orchestra.console.smithi149:power off output: Chassis Power Control: Down/Off
2024-03-11T05:14:45.698 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:14:45.706 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is on
2024-03-11T05:14:45.711 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on
2024-03-11T05:14:45.719 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on
2024-03-11T05:14:49.710 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:14:49.717 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:14:49.722 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:14:49.819 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is on
2024-03-11T05:14:49.824 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on
2024-03-11T05:14:49.831 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on
2024-03-11T05:14:53.824 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:14:53.831 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:14:53.839 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:14:53.933 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is off
2024-03-11T05:14:53.940 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off
2024-03-11T05:14:53.947 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off
2024-03-11T05:14:54.034 INFO:teuthology.orchestra.console.smithi063:Power off completed
2024-03-11T05:14:54.041 INFO:teuthology.orchestra.console.smithi040:Power off completed
2024-03-11T05:14:54.048 INFO:teuthology.orchestra.console.smithi149:Power off completed
2024-03-11T05:14:54.135 INFO:teuthology.orchestra.console.smithi063:Power on
2024-03-11T05:14:54.136 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-11T05:14:54.143 INFO:teuthology.orchestra.console.smithi040:Power on
2024-03-11T05:14:54.143 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-11T05:14:54.150 INFO:teuthology.orchestra.console.smithi149:Power on
2024-03-11T05:14:54.150 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-11T05:14:54.157 DEBUG:teuthology.orchestra.console.smithi063:power on output: Chassis Power Control: Up/On
2024-03-11T05:14:54.158 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:14:54.165 DEBUG:teuthology.orchestra.console.smithi040:power on output: Chassis Power Control: Up/On
2024-03-11T05:14:54.165 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:14:54.172 DEBUG:teuthology.orchestra.console.smithi149:power on output: Chassis Power Control: Up/On
2024-03-11T05:14:54.173 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:14:54.181 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is off
2024-03-11T05:14:54.185 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off
2024-03-11T05:14:54.194 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off
2024-03-11T05:14:58.185 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:14:58.192 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:14:58.198 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:14:58.294 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is off
2024-03-11T05:14:58.300 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off
2024-03-11T05:14:58.306 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off
2024-03-11T05:15:02.298 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:15:02.305 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:15:02.311 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T05:15:02.406 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is on
2024-03-11T05:15:02.413 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on
2024-03-11T05:15:02.500 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on
2024-03-11T05:15:02.507 INFO:teuthology.orchestra.console.smithi063:Power on completed
2024-03-11T05:15:02.514 INFO:teuthology.orchestra.console.smithi040:Power on completed
2024-03-11T05:15:02.601 INFO:teuthology.orchestra.console.smithi149:Power on completed
2024-03-11T05:15:02.608 INFO:teuthology.provision.fog.smithi063:Waiting for deploy to finish
2024-03-11T05:15:02.615 INFO:teuthology.provision.fog.smithi040:Waiting for deploy to finish
2024-03-11T05:15:02.702 INFO:teuthology.provision.fog.smithi149:Waiting for deploy to finish
2024-03-11T05:17:33.406 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:17:36.478 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40
2024-03-11T05:17:42.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:17:45.534 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40
2024-03-11T05:17:48.493 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:17:48.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:17:51.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:17:51.554 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2024-03-11T05:17:51.554 WARNING:teuthology.provision.fog.smithi063:[Errno None] Unable to connect to port 22 on 172.21.15.63
2024-03-11T05:17:54.594 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40
2024-03-11T05:17:57.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:17:57.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:18:00.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:18:00.642 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2024-03-11T05:18:00.642 WARNING:teuthology.provision.fog.smithi063:[Errno None] Unable to connect to port 22 on 172.21.15.63
2024-03-11T05:18:03.678 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40
2024-03-11T05:18:06.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:18:06.648 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:18:09.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:18:09.726 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2024-03-11T05:18:09.726 WARNING:teuthology.provision.fog.smithi063:[Errno None] Unable to connect to port 22 on 172.21.15.63
2024-03-11T05:18:12.734 WARNING:teuthology.provision.fog.smithi040:[Errno None] Unable to connect to port 22 on 172.21.15.40
2024-03-11T05:18:15.730 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:18:15.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:18:18.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:18:18.782 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2024-03-11T05:18:18.783 WARNING:teuthology.provision.fog.smithi063:[Errno None] Unable to connect to port 22 on 172.21.15.63
2024-03-11T05:18:24.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:18:24.789 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:18:27.838 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2024-03-11T05:18:27.870 WARNING:teuthology.provision.fog.smithi063:[Errno None] Unable to connect to port 22 on 172.21.15.63
2024-03-11T05:18:33.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:18:33.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:18:36.894 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2024-03-11T05:18:36.926 WARNING:teuthology.provision.fog.smithi063:[Errno None] Unable to connect to port 22 on 172.21.15.63
2024-03-11T05:18:42.900 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:18:42.927 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:18:45.950 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2024-03-11T05:18:51.958 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:19:01.406 WARNING:teuthology.provision.fog.smithi063:[Errno None] Unable to connect to port 22 on 172.21.15.63
2024-03-11T05:19:07.414 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:19:10.498 WARNING:teuthology.provision.fog.smithi063:[Errno None] Unable to connect to port 22 on 172.21.15.63
2024-03-11T05:19:16.504 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:19:18.743 WARNING:teuthology.provision.fog.smithi040:timed out
2024-03-11T05:19:19.582 WARNING:teuthology.provision.fog.smithi063:[Errno None] Unable to connect to port 22 on 172.21.15.63
2024-03-11T05:19:24.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:19:25.092 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-11T05:19:25.093 DEBUG:teuthology.orchestra.run.smithi040:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-11T05:19:25.428 INFO:teuthology.provision.fog.smithi040:Node is ready
2024-03-11T05:19:25.428 DEBUG:teuthology.orchestra.run.smithi040:> hostname
2024-03-11T05:19:25.443 INFO:teuthology.orchestra.run.smithi040.stdout:smithi040.front.sepia.ceph.com
2024-03-11T05:19:25.443 DEBUG:teuthology.orchestra.run.smithi040:> grep smithi040.front.sepia.ceph.com /etc/hosts
2024-03-11T05:19:25.498 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40 smithi040.front.sepia.ceph.com smithi040
2024-03-11T05:19:25.498 DEBUG:teuthology.orchestra.run.smithi040:> sudo hostname smithi040
2024-03-11T05:19:25.584 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:19:25.612 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i -e 's/smithi040.front.sepia.ceph.com/smithi040/g' /etc/hosts
2024-03-11T05:19:25.672 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i -e 's/172.21.15.40/172.21.15.40/g' /etc/hosts
2024-03-11T05:19:25.743 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i -e 's/smithi040.front.sepia.ceph.com/smithi040/g' /etc/hostname
2024-03-11T05:19:25.814 DEBUG:teuthology.orchestra.run.smithi040:> sudo hostname smithi040
2024-03-11T05:19:25.866 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-11T05:19:25.866 DEBUG:teuthology.orchestra.run.smithi063:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-11T05:19:25.884 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release
2024-03-11T05:19:25.939 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream"
2024-03-11T05:19:25.939 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="9"
2024-03-11T05:19:25.939 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos"
2024-03-11T05:19:25.940 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora"
2024-03-11T05:19:25.940 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="9"
2024-03-11T05:19:25.940 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el9"
2024-03-11T05:19:25.940 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-11T05:19:25.940 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31"
2024-03-11T05:19:25.940 INFO:teuthology.orchestra.run.smithi040.stdout:LOGO="fedora-logo-icon"
2024-03-11T05:19:25.940 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-11T05:19:25.940 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/"
2024-03-11T05:19:25.940 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-11T05:19:25.940 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-11T05:19:25.940 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-11T05:19:25.941 INFO:teuthology.provision.fog.smithi040:Deploy complete!
2024-03-11T05:19:25.943 DEBUG:teuthology.parallel:result is None
2024-03-11T05:19:46.220 INFO:teuthology.provision.fog.smithi063:Node is ready
2024-03-11T05:19:46.220 DEBUG:teuthology.orchestra.run.smithi063:> hostname
2024-03-11T05:19:46.239 INFO:teuthology.orchestra.run.smithi063.stdout:smithi063.front.sepia.ceph.com
2024-03-11T05:19:46.240 DEBUG:teuthology.orchestra.run.smithi063:> grep smithi063.front.sepia.ceph.com /etc/hosts
2024-03-11T05:19:46.298 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63 smithi063.front.sepia.ceph.com smithi063
2024-03-11T05:19:46.298 DEBUG:teuthology.orchestra.run.smithi063:> sudo hostname smithi063
2024-03-11T05:19:46.438 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i -e 's/smithi063.front.sepia.ceph.com/smithi063/g' /etc/hosts
2024-03-11T05:19:46.472 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i -e 's/172.21.15.63/172.21.15.63/g' /etc/hosts
2024-03-11T05:19:46.546 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i -e 's/smithi063.front.sepia.ceph.com/smithi063/g' /etc/hostname
2024-03-11T05:19:46.619 DEBUG:teuthology.orchestra.run.smithi063:> sudo hostname smithi063
2024-03-11T05:19:46.690 DEBUG:teuthology.orchestra.run.smithi063:> cat /etc/os-release
2024-03-11T05:19:46.747 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="CentOS Stream"
2024-03-11T05:19:46.747 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="9"
2024-03-11T05:19:46.747 INFO:teuthology.orchestra.run.smithi063.stdout:ID="centos"
2024-03-11T05:19:46.747 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE="rhel fedora"
2024-03-11T05:19:46.747 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="9"
2024-03-11T05:19:46.747 INFO:teuthology.orchestra.run.smithi063.stdout:PLATFORM_ID="platform:el9"
2024-03-11T05:19:46.747 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-11T05:19:46.747 INFO:teuthology.orchestra.run.smithi063.stdout:ANSI_COLOR="0;31"
2024-03-11T05:19:46.747 INFO:teuthology.orchestra.run.smithi063.stdout:LOGO="fedora-logo-icon"
2024-03-11T05:19:46.747 INFO:teuthology.orchestra.run.smithi063.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-11T05:19:46.747 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://centos.org/"
2024-03-11T05:19:46.748 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-11T05:19:46.748 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-11T05:19:46.748 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-11T05:19:46.748 INFO:teuthology.provision.fog.smithi063:Deploy complete!
2024-03-11T05:19:46.749 DEBUG:teuthology.parallel:result is None
2024-03-11T05:19:51.965 WARNING:teuthology.provision.fog.smithi149:timed out
2024-03-11T05:19:57.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:19:58.229 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-11T05:19:58.229 DEBUG:teuthology.orchestra.run.smithi149:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-11T05:19:58.541 INFO:teuthology.provision.fog.smithi149:Node is ready
2024-03-11T05:19:58.541 DEBUG:teuthology.orchestra.run.smithi149:> hostname
2024-03-11T05:19:58.555 INFO:teuthology.orchestra.run.smithi149.stdout:smithi149.front.sepia.ceph.com
2024-03-11T05:19:58.555 DEBUG:teuthology.orchestra.run.smithi149:> grep smithi149.front.sepia.ceph.com /etc/hosts
2024-03-11T05:19:58.611 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.15.149 smithi149.front.sepia.ceph.com smithi149
2024-03-11T05:19:58.611 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname smithi149
2024-03-11T05:19:58.710 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/smithi149.front.sepia.ceph.com/smithi149/g' /etc/hosts
2024-03-11T05:19:58.738 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/172.21.15.149/172.21.15.149/g' /etc/hosts
2024-03-11T05:19:58.805 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/smithi149.front.sepia.ceph.com/smithi149/g' /etc/hostname
2024-03-11T05:19:58.873 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname smithi149
2024-03-11T05:19:58.939 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release
2024-03-11T05:19:58.995 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream"
2024-03-11T05:19:58.995 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9"
2024-03-11T05:19:58.995 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos"
2024-03-11T05:19:58.995 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora"
2024-03-11T05:19:58.995 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9"
2024-03-11T05:19:58.995 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9"
2024-03-11T05:19:58.996 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-11T05:19:58.996 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31"
2024-03-11T05:19:58.996 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon"
2024-03-11T05:19:58.996 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-11T05:19:58.996 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/"
2024-03-11T05:19:58.996 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-11T05:19:58.996 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-11T05:19:58.996 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-11T05:19:58.997 INFO:teuthology.provision.fog.smithi149:Deploy complete!
2024-03-11T05:19:58.997 DEBUG:teuthology.parallel:result is None
2024-03-11T05:19:59.236 INFO:teuthology.lock.ops:Checking smithi040.front.sepia.ceph.com
2024-03-11T05:19:59.236 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-11T05:19:59.255 INFO:teuthology.lock.ops:Checking smithi063.front.sepia.ceph.com
2024-03-11T05:19:59.255 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-11T05:19:59.271 INFO:teuthology.lock.ops:Checking smithi149.front.sepia.ceph.com
2024-03-11T05:19:59.271 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-11T05:19:59.289 INFO:teuthology.lock.ops:Updating [smithi040.front.sepia.ceph.com]: set os type and version on server
2024-03-11T05:19:59.289 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com'
2024-03-11T05:19:59.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:19:59.391 DEBUG:teuthology.orchestra.run.smithi040:> true
2024-03-11T05:19:59.573 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com'
2024-03-11T05:19:59.573 DEBUG:teuthology.orchestra.run.smithi040:> uname -m
2024-03-11T05:19:59.628 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64
2024-03-11T05:19:59.629 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release
2024-03-11T05:19:59.684 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream"
2024-03-11T05:19:59.684 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="9"
2024-03-11T05:19:59.684 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos"
2024-03-11T05:19:59.684 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora"
2024-03-11T05:19:59.684 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="9"
2024-03-11T05:19:59.684 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el9"
2024-03-11T05:19:59.684 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-11T05:19:59.684 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31"
2024-03-11T05:19:59.685 INFO:teuthology.orchestra.run.smithi040.stdout:LOGO="fedora-logo-icon"
2024-03-11T05:19:59.685 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-11T05:19:59.685 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/"
2024-03-11T05:19:59.685 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-11T05:19:59.685 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-11T05:19:59.685 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-11T05:19:59.685 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server
2024-03-11T05:19:59.713 INFO:teuthology.lock.ops:Updating [smithi063.front.sepia.ceph.com]: set os type and version on server
2024-03-11T05:19:59.713 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com'
2024-03-11T05:19:59.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:19:59.830 DEBUG:teuthology.orchestra.run.smithi063:> true
2024-03-11T05:20:00.017 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com'
2024-03-11T05:20:00.017 DEBUG:teuthology.orchestra.run.smithi063:> uname -m
2024-03-11T05:20:00.076 INFO:teuthology.orchestra.run.smithi063.stdout:x86_64
2024-03-11T05:20:00.076 DEBUG:teuthology.orchestra.run.smithi063:> cat /etc/os-release
2024-03-11T05:20:00.134 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="CentOS Stream"
2024-03-11T05:20:00.135 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="9"
2024-03-11T05:20:00.135 INFO:teuthology.orchestra.run.smithi063.stdout:ID="centos"
2024-03-11T05:20:00.135 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE="rhel fedora"
2024-03-11T05:20:00.135 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="9"
2024-03-11T05:20:00.135 INFO:teuthology.orchestra.run.smithi063.stdout:PLATFORM_ID="platform:el9"
2024-03-11T05:20:00.135 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-11T05:20:00.135 INFO:teuthology.orchestra.run.smithi063.stdout:ANSI_COLOR="0;31"
2024-03-11T05:20:00.135 INFO:teuthology.orchestra.run.smithi063.stdout:LOGO="fedora-logo-icon"
2024-03-11T05:20:00.135 INFO:teuthology.orchestra.run.smithi063.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-11T05:20:00.135 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://centos.org/"
2024-03-11T05:20:00.135 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-11T05:20:00.135 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-11T05:20:00.136 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-11T05:20:00.136 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server
2024-03-11T05:20:00.155 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: set os type and version on server
2024-03-11T05:20:00.155 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com'
2024-03-11T05:20:00.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T05:20:00.244 DEBUG:teuthology.orchestra.run.smithi149:> true
2024-03-11T05:20:00.314 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com'
2024-03-11T05:20:00.315 DEBUG:teuthology.orchestra.run.smithi149:> uname -m
2024-03-11T05:20:00.371 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64
2024-03-11T05:20:00.371 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release
2024-03-11T05:20:00.427 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream"
2024-03-11T05:20:00.427 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9"
2024-03-11T05:20:00.427 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos"
2024-03-11T05:20:00.427 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora"
2024-03-11T05:20:00.427 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9"
2024-03-11T05:20:00.427 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9"
2024-03-11T05:20:00.427 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-11T05:20:00.427 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31"
2024-03-11T05:20:00.427 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon"
2024-03-11T05:20:00.427 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-11T05:20:00.428 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/"
2024-03-11T05:20:00.428 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-11T05:20:00.428 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-11T05:20:00.428 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-11T05:20:00.428 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server
2024-03-11T05:20:00.447 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:20:00.542 INFO:teuthology.dispatcher.supervisor:Running job 7590707
2024-03-11T05:20:00.542 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-03-11_04:49:03-fs-main-testing-default-smithi/7590707 --name vshankar-2024-03-11_04:49:03-fs-main-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/postgres}} -- /home/teuthworker/archive/vshankar-2024-03-11_04:49:03-fs-main-testing-default-smithi/7590707/orig.config.yaml
2024-03-11T05:20:00.546 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-03-11_04:49:03-fs-main-testing-default-smithi/7590707
2024-03-11T05:20:00.546 INFO:teuthology.dispatcher.supervisor:Job PID: 48984
2024-03-11T05:20:00.546 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-03-11T05:22:00.615 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:24:00.757 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:26:00.935 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:28:01.099 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:30:01.279 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:32:01.451 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:34:01.637 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:36:01.799 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:38:01.959 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:40:02.152 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:42:02.327 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:44:02.507 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:46:02.681 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:48:02.853 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:50:03.019 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:52:03.188 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:54:03.359 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:56:03.529 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T05:58:03.719 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T06:00:03.880 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T06:02:04.127 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T06:04:04.301 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T06:06:04.467 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T06:06:04.546 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1