2023-11-09T03:02:06.194 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2
2023-11-09T03:02:06.197 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T03:02:06.350 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fa5da26fbb0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fa5d9f22d30>, signals=[15])
2023-11-09T03:02:06.352 INFO:teuthology.lock.ops:Start node 'smithi098.front.sepia.ceph.com' reimaging
2023-11-09T03:02:06.352 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: reset os type and version on server
2023-11-09T03:02:06.352 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server
2023-11-09T03:02:06.371 INFO:teuthology.lock.ops:Start node 'smithi104.front.sepia.ceph.com' reimaging
2023-11-09T03:02:06.372 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: reset os type and version on server
2023-11-09T03:02:06.372 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server
2023-11-09T03:02:06.373 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /host {"name": "smithi098"}
2023-11-09T03:02:06.396 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /host {"name": "smithi104"}
2023-11-09T03:02:06.442 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-11-09T03:02:06.472 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-11-09T03:02:06.487 DEBUG:teuthology.provision.fog.smithi098:Requesting image smithi_rhel_8.6 (ID 5)
2023-11-09T03:02:06.487 DEBUG:teuthology.provision.fog.smithi098:FOG request: PUT /host/203 {"imageID": 5}
2023-11-09T03:02:06.525 DEBUG:teuthology.provision.fog.smithi104:Requesting image smithi_rhel_8.6 (ID 5)
2023-11-09T03:02:06.525 DEBUG:teuthology.provision.fog.smithi104:FOG request: PUT /host/209 {"imageID": 5}
2023-11-09T03:02:06.530 INFO:teuthology.provision.fog.smithi098:Scheduling deploy of rhel 8.6
2023-11-09T03:02:06.531 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None
2023-11-09T03:02:06.582 INFO:teuthology.provision.fog.smithi104:Scheduling deploy of rhel 8.6
2023-11-09T03:02:06.583 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-11-09T03:02:06.644 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /tasktype {"name": "deploy"}
2023-11-09T03:02:06.664 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /tasktype {"name": "deploy"}
2023-11-09T03:02:06.684 DEBUG:teuthology.provision.fog.smithi098:FOG request: POST /host/203/task {"taskTypeID": 1}
2023-11-09T03:02:06.702 DEBUG:teuthology.provision.fog.smithi104:FOG request: POST /host/209/task {"taskTypeID": 1}
2023-11-09T03:02:06.747 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None
2023-11-09T03:02:06.786 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-11-09T03:02:06.894 INFO:teuthology.orchestra.console.smithi098:Power off
2023-11-09T03:02:06.895 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-09T03:02:06.922 INFO:teuthology.orchestra.console.smithi104:Power off
2023-11-09T03:02:06.923 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-09T03:02:06.936 DEBUG:teuthology.orchestra.console.smithi098:power off output: Chassis Power Control: Down/Off
2023-11-09T03:02:06.937 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T03:02:06.951 DEBUG:teuthology.orchestra.console.smithi104:power off output: Chassis Power Control: Down/Off
2023-11-09T03:02:06.951 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T03:02:06.960 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off
2023-11-09T03:02:06.977 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is off
2023-11-09T03:02:07.062 INFO:teuthology.orchestra.console.smithi098:Power off completed
2023-11-09T03:02:07.079 INFO:teuthology.orchestra.console.smithi104:Power off completed
2023-11-09T03:02:07.162 INFO:teuthology.orchestra.console.smithi098:Power on
2023-11-09T03:02:07.163 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-09T03:02:07.180 INFO:teuthology.orchestra.console.smithi104:Power on
2023-11-09T03:02:07.181 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-09T03:02:07.190 DEBUG:teuthology.orchestra.console.smithi098:power on output: Chassis Power Control: Up/On
2023-11-09T03:02:07.190 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T03:02:07.212 DEBUG:teuthology.orchestra.console.smithi104:power on output: Chassis Power Control: Up/On
2023-11-09T03:02:07.213 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T03:02:07.221 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off
2023-11-09T03:02:07.235 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is off
2023-11-09T03:02:11.226 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T03:02:11.239 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T03:02:11.342 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off
2023-11-09T03:02:11.351 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is off
2023-11-09T03:02:15.348 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T03:02:15.363 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T03:02:15.473 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on
2023-11-09T03:02:15.550 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is on
2023-11-09T03:02:15.575 INFO:teuthology.orchestra.console.smithi104:Power on completed
2023-11-09T03:02:15.652 INFO:teuthology.orchestra.console.smithi098:Power on completed
2023-11-09T03:02:15.676 INFO:teuthology.provision.fog.smithi104:Waiting for deploy to finish
2023-11-09T03:02:15.677 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-11-09T03:02:15.753 INFO:teuthology.provision.fog.smithi098:Waiting for deploy to finish
2023-11-09T03:02:15.753 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None
2023-11-09T03:02:30.802 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-11-09T03:02:30.851 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None
2023-11-09T03:02:45.903 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-11-09T03:02:45.941 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None
2023-11-09T03:03:01.003 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-11-09T03:03:01.011 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None
2023-11-09T03:03:16.094 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None
2023-11-09T03:03:16.096 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-11-09T03:03:31.194 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-11-09T03:03:31.196 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None
2023-11-09T03:03:46.310 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None
2023-11-09T03:03:46.312 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-11-09T03:04:01.403 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-11-09T03:04:01.405 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None
2023-11-09T03:04:16.500 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None
2023-11-09T03:04:16.501 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-11-09T03:04:31.594 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-11-09T03:04:31.596 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None
2023-11-09T03:04:46.686 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-11-09T03:04:46.688 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None
2023-11-09T03:04:46.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T03:04:46.862 ERROR:teuthology.orchestra.connection:Error authenticating with smithi104.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-09T03:05:01.767 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None
2023-11-09T03:05:01.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T03:05:04.926 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-11-09T03:05:10.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T03:05:13.982 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-11-09T03:05:19.984 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T03:05:23.070 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-11-09T03:05:29.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T03:05:32.154 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-11-09T03:05:38.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T03:05:41.214 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-11-09T03:05:47.216 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T03:05:50.298 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-11-09T03:05:50.868 WARNING:teuthology.provision.fog.smithi104:timed out
2023-11-09T03:05:56.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T03:05:56.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T03:06:14.779 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-11-09T03:06:15.290 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104
2023-11-09T03:06:20.781 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T03:06:21.292 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T03:06:23.838 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-11-09T03:06:24.401 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-09T03:06:24.402 DEBUG:teuthology.orchestra.run.smithi104:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-09T03:06:24.671 INFO:teuthology.provision.fog.smithi104:Node is ready
2023-11-09T03:06:24.671 DEBUG:teuthology.orchestra.run.smithi104:> hostname
2023-11-09T03:06:24.687 INFO:teuthology.orchestra.run.smithi104.stdout:smithi022
2023-11-09T03:06:24.688 DEBUG:teuthology.orchestra.run.smithi104:> grep smithi022 /etc/hosts
2023-11-09T03:06:24.744 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022
2023-11-09T03:06:24.744 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname smithi104
2023-11-09T03:06:24.855 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/smithi022/smithi104/g' /etc/hosts
2023-11-09T03:06:24.882 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/172.21.15.22/172.21.15.104/g' /etc/hosts
2023-11-09T03:06:24.950 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/smithi022/smithi104/g' /etc/hostname
2023-11-09T03:06:25.016 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname smithi104
2023-11-09T03:06:25.082 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release
2023-11-09T03:06:25.138 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="Red Hat Enterprise Linux"
2023-11-09T03:06:25.138 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="8.6 (Ootpa)"
2023-11-09T03:06:25.138 INFO:teuthology.orchestra.run.smithi104.stdout:ID="rhel"
2023-11-09T03:06:25.138 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="fedora"
2023-11-09T03:06:25.139 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="8.6"
2023-11-09T03:06:25.139 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el8"
2023-11-09T03:06:25.139 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-11-09T03:06:25.139 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31"
2023-11-09T03:06:25.140 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-11-09T03:06:25.140 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://www.redhat.com/"
2023-11-09T03:06:25.140 INFO:teuthology.orchestra.run.smithi104.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-09T03:06:25.140 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-09T03:06:25.140 INFO:teuthology.orchestra.run.smithi104.stdout:
2023-11-09T03:06:25.141 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-09T03:06:25.141 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-11-09T03:06:25.141 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-09T03:06:25.141 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-11-09T03:06:25.142 INFO:teuthology.provision.fog.smithi104:Deploy complete!
2023-11-09T03:06:25.143 DEBUG:teuthology.parallel:result is None
2023-11-09T03:06:29.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T03:06:32.894 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98
2023-11-09T03:06:38.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T03:06:39.246 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-09T03:06:39.246 DEBUG:teuthology.orchestra.run.smithi098:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-09T03:06:39.497 INFO:teuthology.provision.fog.smithi098:Node is ready
2023-11-09T03:06:39.498 DEBUG:teuthology.orchestra.run.smithi098:> hostname
2023-11-09T03:06:39.514 INFO:teuthology.orchestra.run.smithi098.stdout:smithi098.front.sepia.ceph.com
2023-11-09T03:06:39.515 DEBUG:teuthology.orchestra.run.smithi098:> grep smithi098.front.sepia.ceph.com /etc/hosts
2023-11-09T03:06:39.571 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98 smithi098.front.sepia.ceph.com smithi098
2023-11-09T03:06:39.571 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname smithi098
2023-11-09T03:06:39.715 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/smithi098.front.sepia.ceph.com/smithi098/g' /etc/hosts
2023-11-09T03:06:39.743 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/172.21.15.98/172.21.15.98/g' /etc/hosts
2023-11-09T03:06:39.809 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/smithi098.front.sepia.ceph.com/smithi098/g' /etc/hostname
2023-11-09T03:06:39.876 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname smithi098
2023-11-09T03:06:39.942 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release
2023-11-09T03:06:39.997 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="Red Hat Enterprise Linux"
2023-11-09T03:06:39.998 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8.6 (Ootpa)"
2023-11-09T03:06:39.998 INFO:teuthology.orchestra.run.smithi098.stdout:ID="rhel"
2023-11-09T03:06:39.998 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="fedora"
2023-11-09T03:06:39.999 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8.6"
2023-11-09T03:06:39.999 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8"
2023-11-09T03:06:39.999 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-11-09T03:06:39.999 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31"
2023-11-09T03:06:39.999 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-11-09T03:06:40.000 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://www.redhat.com/"
2023-11-09T03:06:40.000 INFO:teuthology.orchestra.run.smithi098.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-09T03:06:40.000 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-09T03:06:40.000 INFO:teuthology.orchestra.run.smithi098.stdout:
2023-11-09T03:06:40.000 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-09T03:06:40.000 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-11-09T03:06:40.001 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-09T03:06:40.001 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-11-09T03:06:40.001 INFO:teuthology.provision.fog.smithi098:Deploy complete!
2023-11-09T03:06:40.002 DEBUG:teuthology.parallel:result is None
2023-11-09T03:06:40.155 INFO:teuthology.lock.ops:Checking smithi098.front.sepia.ceph.com
2023-11-09T03:06:40.156 INFO:teuthology.lock.ops:Checking smithi104.front.sepia.ceph.com
2023-11-09T03:06:40.157 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: set os type and version on server
2023-11-09T03:06:40.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-11-09T03:06:40.158 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T03:06:40.225 DEBUG:teuthology.orchestra.run.smithi098:> true
2023-11-09T03:06:40.296 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com'
2023-11-09T03:06:40.296 DEBUG:teuthology.orchestra.run.smithi098:> uname -m
2023-11-09T03:06:40.352 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64
2023-11-09T03:06:40.352 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release
2023-11-09T03:06:40.409 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="Red Hat Enterprise Linux"
2023-11-09T03:06:40.409 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8.6 (Ootpa)"
2023-11-09T03:06:40.410 INFO:teuthology.orchestra.run.smithi098.stdout:ID="rhel"
2023-11-09T03:06:40.410 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="fedora"
2023-11-09T03:06:40.410 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8.6"
2023-11-09T03:06:40.410 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8"
2023-11-09T03:06:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-11-09T03:06:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31"
2023-11-09T03:06:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-11-09T03:06:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://www.redhat.com/"
2023-11-09T03:06:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-09T03:06:40.412 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-09T03:06:40.412 INFO:teuthology.orchestra.run.smithi098.stdout:
2023-11-09T03:06:40.412 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-09T03:06:40.412 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-11-09T03:06:40.412 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-09T03:06:40.413 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-11-09T03:06:40.413 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server
2023-11-09T03:06:40.435 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: set os type and version on server
2023-11-09T03:06:40.436 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com'
2023-11-09T03:06:40.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T03:06:40.503 DEBUG:teuthology.orchestra.run.smithi104:> true
2023-11-09T03:06:40.593 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com'
2023-11-09T03:06:40.594 DEBUG:teuthology.orchestra.run.smithi104:> uname -m
2023-11-09T03:06:40.651 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64
2023-11-09T03:06:40.652 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release
2023-11-09T03:06:40.708 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="Red Hat Enterprise Linux"
2023-11-09T03:06:40.708 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="8.6 (Ootpa)"
2023-11-09T03:06:40.709 INFO:teuthology.orchestra.run.smithi104.stdout:ID="rhel"
2023-11-09T03:06:40.709 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="fedora"
2023-11-09T03:06:40.709 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="8.6"
2023-11-09T03:06:40.709 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el8"
2023-11-09T03:06:40.709 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-11-09T03:06:40.710 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31"
2023-11-09T03:06:40.710 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-11-09T03:06:40.710 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://www.redhat.com/"
2023-11-09T03:06:40.710 INFO:teuthology.orchestra.run.smithi104.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-09T03:06:40.711 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-09T03:06:40.711 INFO:teuthology.orchestra.run.smithi104.stdout:
2023-11-09T03:06:40.711 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-09T03:06:40.711 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-11-09T03:06:40.711 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-09T03:06:40.712 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-11-09T03:06:40.712 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server
2023-11-09T03:06:40.735 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T03:06:40.836 INFO:teuthology.dispatcher.supervisor:Running job 7452399
2023-11-09T03:06:40.836 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2023-11-09_02:25:01-upgrade:reef-x-main-distro-default-smithi/7452399 --name teuthology-2023-11-09_02:25:01-upgrade:reef-x-main-distro-default-smithi --description upgrade:reef-x/parallel/{0-random-distro$/{rhel_8.6_container_tools_3.0} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} -- /home/teuthworker/archive/teuthology-2023-11-09_02:25:01-upgrade:reef-x-main-distro-default-smithi/7452399/orig.config.yaml
2023-11-09T03:06:40.840 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2023-11-09_02:25:01-upgrade:reef-x-main-distro-default-smithi/7452399
2023-11-09T03:06:40.841 INFO:teuthology.dispatcher.supervisor:Job PID: 448435
2023-11-09T03:06:40.841 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-11-09T03:08:40.843 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T03:10:40.916 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T03:12:40.982 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T03:14:41.051 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T03:16:41.117 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T03:18:41.186 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T03:20:41.266 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T03:22:41.332 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T03:22:41.400 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-11-09T03:22:41.521 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-11-09T03:22:41.522 INFO:teuthology.nuke:Checking targets against current locks
2023-11-09T03:22:41.569 DEBUG:teuthology.nuke:shortname: smithi098
2023-11-09T03:22:41.570 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-09T03:22:41.589 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2023-11-09_02:25:01-upgrade:reef-x-main-distro-default-smithi/7452399', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-09 03:02:05.354164', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}
2023-11-09T03:22:41.596 DEBUG:teuthology.nuke:shortname: smithi104
2023-11-09T03:22:41.597 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-09T03:22:41.616 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2023-11-09_02:25:01-upgrade:reef-x-main-distro-default-smithi/7452399', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-09 03:02:05.352180', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}
2023-11-09T03:22:41.633 INFO:teuthology.orchestra.console.smithi098:Power off
2023-11-09T03:22:41.634 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-09T03:22:41.657 DEBUG:teuthology.orchestra.console.smithi098:power off output: Chassis Power Control: Down/Off
2023-11-09T03:22:41.658 DEBUG:teuthology.orchestra.console.smithi098:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T03:22:41.673 INFO:teuthology.orchestra.console.smithi104:Power off
2023-11-09T03:22:41.674 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-09T03:22:41.683 DEBUG:teuthology.orchestra.console.smithi098:check power output: Chassis Power is off
2023-11-09T03:22:41.697 DEBUG:teuthology.orchestra.console.smithi104:power off output: Chassis Power Control: Down/Off
2023-11-09T03:22:41.697 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T03:22:41.721 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is off
2023-11-09T03:22:41.784 INFO:teuthology.orchestra.console.smithi098:Power off completed
2023-11-09T03:22:41.822 INFO:teuthology.orchestra.console.smithi104:Power off completed
2023-11-09T03:22:41.934 INFO:teuthology.lock.ops:unlocked: smithi098.front.sepia.ceph.com
2023-11-09T03:22:41.965 INFO:teuthology.lock.ops:unlocked: smithi104.front.sepia.ceph.com