2023-09-23T16:30:10.021 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-09-23T16:30:10.021 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T16:30:10.158 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f416a70caf0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f416a44fee0>, signals=[15])
2023-09-23T16:30:10.159 INFO:teuthology.lock.ops:Start node 'smithi159.front.sepia.ceph.com' reimaging
2023-09-23T16:30:10.159 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: reset os type and version on server
2023-09-23T16:30:10.159 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server
2023-09-23T16:30:10.180 INFO:teuthology.lock.ops:Start node 'smithi164.front.sepia.ceph.com' reimaging
2023-09-23T16:30:10.180 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: reset os type and version on server
2023-09-23T16:30:10.180 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server
2023-09-23T16:30:10.181 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /host {"name": "smithi159"}
2023-09-23T16:30:10.205 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /host {"name": "smithi164"}
2023-09-23T16:30:10.257 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-09-23T16:30:10.273 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-09-23T16:30:10.299 DEBUG:teuthology.provision.fog.smithi159:Requesting image smithi_rhel_8.6 (ID 5)
2023-09-23T16:30:10.299 DEBUG:teuthology.provision.fog.smithi159:FOG request: PUT /host/264 {"imageID": 5}
2023-09-23T16:30:10.323 DEBUG:teuthology.provision.fog.smithi164:Requesting image smithi_rhel_8.6 (ID 5)
2023-09-23T16:30:10.324 DEBUG:teuthology.provision.fog.smithi164:FOG request: PUT /host/269 {"imageID": 5}
2023-09-23T16:30:10.343 INFO:teuthology.provision.fog.smithi159:Scheduling deploy of rhel 8.6
2023-09-23T16:30:10.343 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None
2023-09-23T16:30:10.379 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of rhel 8.6
2023-09-23T16:30:10.379 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-09-23T16:30:10.399 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /tasktype {"name": "deploy"}
2023-09-23T16:30:10.425 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /tasktype {"name": "deploy"}
2023-09-23T16:30:10.433 DEBUG:teuthology.provision.fog.smithi159:FOG request: POST /host/264/task {"taskTypeID": 1}
2023-09-23T16:30:10.458 DEBUG:teuthology.provision.fog.smithi164:FOG request: POST /host/269/task {"taskTypeID": 1}
2023-09-23T16:30:10.490 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None
2023-09-23T16:30:10.517 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-09-23T16:30:10.574 INFO:teuthology.orchestra.console.smithi159:Power off
2023-09-23T16:30:10.574 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-23T16:30:10.589 INFO:teuthology.orchestra.console.smithi164:Power off
2023-09-23T16:30:10.590 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-23T16:30:10.598 DEBUG:teuthology.orchestra.console.smithi159:power off output: Chassis Power Control: Down/Off
2023-09-23T16:30:10.598 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-23T16:30:10.611 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off
2023-09-23T16:30:10.612 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-23T16:30:10.619 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on
2023-09-23T16:30:10.633 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2023-09-23T16:30:14.622 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-23T16:30:14.633 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-23T16:30:14.730 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on
2023-09-23T16:30:14.741 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2023-09-23T16:30:18.735 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-23T16:30:18.741 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-23T16:30:18.843 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off
2023-09-23T16:30:18.850 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2023-09-23T16:30:18.945 INFO:teuthology.orchestra.console.smithi159:Power off completed
2023-09-23T16:30:19.045 INFO:teuthology.orchestra.console.smithi159:Power on
2023-09-23T16:30:19.046 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-23T16:30:19.070 DEBUG:teuthology.orchestra.console.smithi159:power on output: Chassis Power Control: Up/On
2023-09-23T16:30:19.070 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-23T16:30:19.090 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off
2023-09-23T16:30:22.854 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-23T16:30:22.963 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2023-09-23T16:30:23.064 INFO:teuthology.orchestra.console.smithi164:Power off completed
2023-09-23T16:30:23.091 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-23T16:30:23.165 INFO:teuthology.orchestra.console.smithi164:Power on
2023-09-23T16:30:23.165 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-23T16:30:23.185 DEBUG:teuthology.orchestra.console.smithi164:power on output: Chassis Power Control: Up/On
2023-09-23T16:30:23.185 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-23T16:30:23.199 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off
2023-09-23T16:30:23.204 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2023-09-23T16:30:27.202 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-23T16:30:27.209 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-23T16:30:27.312 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on
2023-09-23T16:30:27.317 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2023-09-23T16:30:27.413 INFO:teuthology.orchestra.console.smithi159:Power on completed
2023-09-23T16:30:27.514 INFO:teuthology.provision.fog.smithi159:Waiting for deploy to finish
2023-09-23T16:30:27.514 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None
2023-09-23T16:30:31.319 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-23T16:30:31.427 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2023-09-23T16:30:31.528 INFO:teuthology.orchestra.console.smithi164:Power on completed
2023-09-23T16:30:31.629 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish
2023-09-23T16:30:31.629 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-09-23T16:30:42.582 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None
2023-09-23T16:30:46.694 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-09-23T16:30:57.650 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None
2023-09-23T16:31:01.750 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-09-23T16:31:12.708 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None
2023-09-23T16:31:16.806 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-09-23T16:31:27.774 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None
2023-09-23T16:31:31.866 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-09-23T16:31:42.838 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None
2023-09-23T16:31:46.922 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-09-23T16:31:57.918 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None
2023-09-23T16:32:01.986 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-09-23T16:32:12.990 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None
2023-09-23T16:32:17.054 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-09-23T16:32:28.074 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None
2023-09-23T16:32:32.124 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-09-23T16:32:43.154 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None
2023-09-23T16:32:47.202 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-09-23T16:32:58.236 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None
2023-09-23T16:33:02.266 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-09-23T16:33:13.314 DEBUG:teuthology.provision.fog.smithi159:FOG request: GET /task/active None
2023-09-23T16:33:13.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:33:17.333 DEBUG:teuthology.provision.fog.smithi164:FOG request: GET /task/active None
2023-09-23T16:33:17.402 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:33:31.770 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2023-09-23T16:33:35.866 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-09-23T16:33:37.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:33:40.826 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2023-09-23T16:33:41.868 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:33:44.922 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-09-23T16:33:46.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:33:49.882 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2023-09-23T16:33:50.925 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:33:53.982 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-09-23T16:33:55.885 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:33:58.942 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2023-09-23T16:33:59.984 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:34:03.034 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-09-23T16:34:04.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:34:08.026 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2023-09-23T16:34:09.037 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:34:12.090 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-09-23T16:34:14.029 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:34:17.082 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159
2023-09-23T16:34:18.093 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:34:21.146 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-09-23T16:34:23.085 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:34:27.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:35:23.115 WARNING:teuthology.provision.fog.smithi159:timed out
2023-09-23T16:35:27.157 WARNING:teuthology.provision.fog.smithi164:timed out
2023-09-23T16:35:29.119 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:35:29.471 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-09-23T16:35:29.472 DEBUG:teuthology.orchestra.run.smithi159:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-09-23T16:35:29.751 INFO:teuthology.provision.fog.smithi159:Node is ready
2023-09-23T16:35:29.752 DEBUG:teuthology.orchestra.run.smithi159:> hostname
2023-09-23T16:35:29.770 INFO:teuthology.orchestra.run.smithi159.stdout:smithi159
2023-09-23T16:35:29.770 DEBUG:teuthology.orchestra.run.smithi159:> grep smithi159 /etc/hosts
2023-09-23T16:35:29.828 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159 smithi159.front.sepia.ceph.com smithi159
2023-09-23T16:35:29.828 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159
2023-09-23T16:35:29.940 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi159/smithi159/g' /etc/hosts
2023-09-23T16:35:29.967 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/172.21.15.159/172.21.15.159/g' /etc/hosts
2023-09-23T16:35:30.035 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi159/smithi159/g' /etc/hostname
2023-09-23T16:35:30.102 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159
2023-09-23T16:35:30.168 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release
2023-09-23T16:35:30.225 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="Red Hat Enterprise Linux"
2023-09-23T16:35:30.225 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="8.6 (Ootpa)"
2023-09-23T16:35:30.225 INFO:teuthology.orchestra.run.smithi159.stdout:ID="rhel"
2023-09-23T16:35:30.225 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="fedora"
2023-09-23T16:35:30.225 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="8.6"
2023-09-23T16:35:30.225 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el8"
2023-09-23T16:35:30.225 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-09-23T16:35:30.225 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31"
2023-09-23T16:35:30.226 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-09-23T16:35:30.226 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://www.redhat.com/"
2023-09-23T16:35:30.226 INFO:teuthology.orchestra.run.smithi159.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-09-23T16:35:30.226 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-09-23T16:35:30.226 INFO:teuthology.orchestra.run.smithi159.stdout:
2023-09-23T16:35:30.226 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-09-23T16:35:30.226 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-09-23T16:35:30.226 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-09-23T16:35:30.226 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-09-23T16:35:30.227 INFO:teuthology.provision.fog.smithi159:Deploy complete!
2023-09-23T16:35:30.228 DEBUG:teuthology.parallel:result is None
2023-09-23T16:35:33.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:35:33.422 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-09-23T16:35:33.422 DEBUG:teuthology.orchestra.run.smithi164:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-09-23T16:35:33.709 INFO:teuthology.provision.fog.smithi164:Node is ready
2023-09-23T16:35:33.709 DEBUG:teuthology.orchestra.run.smithi164:> hostname
2023-09-23T16:35:33.769 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164.front.sepia.ceph.com
2023-09-23T16:35:33.769 DEBUG:teuthology.orchestra.run.smithi164:> grep smithi164.front.sepia.ceph.com /etc/hosts
2023-09-23T16:35:33.826 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164 smithi164.front.sepia.ceph.com smithi164
2023-09-23T16:35:33.826 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164
2023-09-23T16:35:33.931 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164.front.sepia.ceph.com/smithi164/g' /etc/hosts
2023-09-23T16:35:33.959 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/172.21.15.164/172.21.15.164/g' /etc/hosts
2023-09-23T16:35:34.028 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164.front.sepia.ceph.com/smithi164/g' /etc/hostname
2023-09-23T16:35:34.096 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164
2023-09-23T16:35:34.163 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release
2023-09-23T16:35:34.219 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="Red Hat Enterprise Linux"
2023-09-23T16:35:34.219 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="8.6 (Ootpa)"
2023-09-23T16:35:34.219 INFO:teuthology.orchestra.run.smithi164.stdout:ID="rhel"
2023-09-23T16:35:34.219 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="fedora"
2023-09-23T16:35:34.220 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="8.6"
2023-09-23T16:35:34.220 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el8"
2023-09-23T16:35:34.220 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-09-23T16:35:34.220 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31"
2023-09-23T16:35:34.220 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-09-23T16:35:34.220 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://www.redhat.com/"
2023-09-23T16:35:34.220 INFO:teuthology.orchestra.run.smithi164.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-09-23T16:35:34.220 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-09-23T16:35:34.220 INFO:teuthology.orchestra.run.smithi164.stdout:
2023-09-23T16:35:34.220 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-09-23T16:35:34.220 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-09-23T16:35:34.220 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-09-23T16:35:34.220 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-09-23T16:35:34.221 INFO:teuthology.provision.fog.smithi164:Deploy complete!
2023-09-23T16:35:34.221 DEBUG:teuthology.parallel:result is None
2023-09-23T16:35:34.368 INFO:teuthology.lock.ops:Checking smithi159.front.sepia.ceph.com
2023-09-23T16:35:34.368 INFO:teuthology.lock.ops:New key found. Updating...
2023-09-23T16:35:34.404 INFO:teuthology.lock.ops:Checking smithi164.front.sepia.ceph.com
2023-09-23T16:35:34.404 INFO:teuthology.lock.ops:New key found. Updating...
2023-09-23T16:35:34.421 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: set os type and version on server
2023-09-23T16:35:34.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com'
2023-09-23T16:35:34.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:35:34.492 DEBUG:teuthology.orchestra.run.smithi159:> true
2023-09-23T16:35:34.561 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com'
2023-09-23T16:35:34.561 DEBUG:teuthology.orchestra.run.smithi159:> uname -m
2023-09-23T16:35:34.623 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64
2023-09-23T16:35:34.623 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release
2023-09-23T16:35:34.685 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="Red Hat Enterprise Linux"
2023-09-23T16:35:34.685 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="8.6 (Ootpa)"
2023-09-23T16:35:34.685 INFO:teuthology.orchestra.run.smithi159.stdout:ID="rhel"
2023-09-23T16:35:34.685 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="fedora"
2023-09-23T16:35:34.685 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="8.6"
2023-09-23T16:35:34.685 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el8"
2023-09-23T16:35:34.685 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-09-23T16:35:34.685 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31"
2023-09-23T16:35:34.685 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-09-23T16:35:34.685 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://www.redhat.com/"
2023-09-23T16:35:34.686 INFO:teuthology.orchestra.run.smithi159.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-09-23T16:35:34.686 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-09-23T16:35:34.686 INFO:teuthology.orchestra.run.smithi159.stdout:
2023-09-23T16:35:34.686 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-09-23T16:35:34.686 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-09-23T16:35:34.686 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-09-23T16:35:34.686 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-09-23T16:35:34.686 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server
2023-09-23T16:35:34.712 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: set os type and version on server
2023-09-23T16:35:34.713 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com'
2023-09-23T16:35:34.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-23T16:35:34.778 DEBUG:teuthology.orchestra.run.smithi164:> true
2023-09-23T16:35:34.842 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com'
2023-09-23T16:35:34.843 DEBUG:teuthology.orchestra.run.smithi164:> uname -m
2023-09-23T16:35:34.898 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64
2023-09-23T16:35:34.898 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release
2023-09-23T16:35:34.954 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="Red Hat Enterprise Linux"
2023-09-23T16:35:34.954 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="8.6 (Ootpa)"
2023-09-23T16:35:34.954 INFO:teuthology.orchestra.run.smithi164.stdout:ID="rhel"
2023-09-23T16:35:34.954 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="fedora"
2023-09-23T16:35:34.954 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="8.6"
2023-09-23T16:35:34.954 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el8"
2023-09-23T16:35:34.954 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-09-23T16:35:34.954 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31"
2023-09-23T16:35:34.955 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-09-23T16:35:34.955 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://www.redhat.com/"
2023-09-23T16:35:34.955 INFO:teuthology.orchestra.run.smithi164.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-09-23T16:35:34.955 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-09-23T16:35:34.955 INFO:teuthology.orchestra.run.smithi164.stdout:
2023-09-23T16:35:34.955 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-09-23T16:35:34.955 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-09-23T16:35:34.955 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-09-23T16:35:34.955 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-09-23T16:35:34.956 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server
2023-09-23T16:35:34.988 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T16:35:35.083 INFO:teuthology.dispatcher.supervisor:Running job 7401743
2023-09-23T16:35:35.084 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2023-09-23_14:23:10-upgrade:quincy-x-reef-distro-default-smithi/7401743 --name teuthology-2023-09-23_14:23:10-upgrade:quincy-x-reef-distro-default-smithi --description upgrade:quincy-x/stress-split/{0-distro/rhel_8.6_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} -- /home/teuthworker/archive/teuthology-2023-09-23_14:23:10-upgrade:quincy-x-reef-distro-default-smithi/7401743/orig.config.yaml
2023-09-23T16:35:35.087 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2023-09-23_14:23:10-upgrade:quincy-x-reef-distro-default-smithi/7401743
2023-09-23T16:35:35.087 INFO:teuthology.dispatcher.supervisor:Job PID: 3516327
2023-09-23T16:35:35.087 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-09-23T16:37:35.178 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T16:39:35.347 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T16:41:35.527 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T16:43:35.672 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T16:45:35.836 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T16:47:35.987 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T16:49:36.138 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T16:51:36.291 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T16:53:36.455 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T16:55:36.616 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T16:57:36.764 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T16:59:36.912 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:01:37.025 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:03:37.191 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:05:37.345 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:07:37.523 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:09:37.676 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:11:37.844 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:13:38.015 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:15:38.183 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:17:38.347 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:19:38.559 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:21:38.729 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:23:38.891 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:25:39.055 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:27:39.232 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:29:39.395 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:31:39.560 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:33:39.728 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:35:39.890 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:37:39.963 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:39:40.125 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:41:40.285 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:43:40.440 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:45:40.607 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:47:40.771 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:49:40.936 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:51:41.107 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:53:41.233 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:55:41.357 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:57:41.688 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T17:59:41.866 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T18:01:42.047 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-23T18:01:42.110 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2023-09-23T18:01:42.306 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-09-23T18:01:42.306 INFO:teuthology.nuke:Checking targets against current locks
2023-09-23T18:01:42.354 DEBUG:teuthology.nuke:shortname: smithi159
2023-09-23T18:01:42.355 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-23T18:01:42.371 DEBUG:teuthology.nuke:shortname: smithi164
2023-09-23T18:01:42.371 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-23T18:01:42.380 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2023-09-23_14:23:10-upgrade:quincy-x-reef-distro-default-smithi/7401743', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 16:30:09.217839', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}
2023-09-23T18:01:42.388 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2023-09-23_14:23:10-upgrade:quincy-x-reef-distro-default-smithi/7401743', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 16:30:09.219453', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}
2023-09-23T18:01:42.422 INFO:teuthology.orchestra.console.smithi159:Power off
2023-09-23T18:01:42.422 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-23T18:01:42.434 INFO:teuthology.orchestra.console.smithi164:Power off
2023-09-23T18:01:42.435 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-23T18:01:42.442 DEBUG:teuthology.orchestra.console.smithi159:power off output: Chassis Power Control: Down/Off
2023-09-23T18:01:42.442 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-23T18:01:42.454 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off
2023-09-23T18:01:42.454 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-23T18:01:42.462 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off
2023-09-23T18:01:42.475 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2023-09-23T18:01:42.564 INFO:teuthology.orchestra.console.smithi159:Power off completed
2023-09-23T18:01:42.576 INFO:teuthology.orchestra.console.smithi164:Power off completed
2023-09-23T18:01:42.718 INFO:teuthology.lock.ops:unlocked: smithi159.front.sepia.ceph.com
2023-09-23T18:01:42.726 INFO:teuthology.lock.ops:unlocked: smithi164.front.sepia.ceph.com