2023-11-14T17:44:16.807 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2
2023-11-14T17:44:16.808 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T17:44:16.951 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fba09506af0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fba0a009670>, signals=[15])
2023-11-14T17:44:16.952 INFO:teuthology.lock.ops:Start node 'smithi069.front.sepia.ceph.com' reimaging
2023-11-14T17:44:16.952 INFO:teuthology.lock.ops:Updating [smithi069.front.sepia.ceph.com]: reset os type and version on server
2023-11-14T17:44:16.953 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server
2023-11-14T17:44:16.974 INFO:teuthology.lock.ops:Start node 'smithi196.front.sepia.ceph.com' reimaging
2023-11-14T17:44:16.975 INFO:teuthology.lock.ops:Updating [smithi196.front.sepia.ceph.com]: reset os type and version on server
2023-11-14T17:44:16.975 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server
2023-11-14T17:44:16.976 DEBUG:teuthology.provision.fog.smithi069:FOG request: GET /host {"name": "smithi069"}
2023-11-14T17:44:17.056 DEBUG:teuthology.provision.fog.smithi069:FOG request: GET /image {"name": "smithi_rhel_8.4"}
2023-11-14T17:44:17.065 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /host {"name": "smithi196"}
2023-11-14T17:44:17.102 DEBUG:teuthology.provision.fog.smithi069:Requesting image smithi_rhel_8.4 (ID 7)
2023-11-14T17:44:17.102 DEBUG:teuthology.provision.fog.smithi069:FOG request: PUT /host/174 {"imageID": 7}
2023-11-14T17:44:17.136 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /image {"name": "smithi_rhel_8.4"}
2023-11-14T17:44:17.146 INFO:teuthology.provision.fog.smithi069:Scheduling deploy of rhel 8.4
2023-11-14T17:44:17.147 DEBUG:teuthology.provision.fog.smithi069:FOG request: GET /task/active None
2023-11-14T17:44:17.179 DEBUG:teuthology.provision.fog.smithi196:Requesting image smithi_rhel_8.4 (ID 7)
2023-11-14T17:44:17.180 DEBUG:teuthology.provision.fog.smithi196:FOG request: PUT /host/301 {"imageID": 7}
2023-11-14T17:44:17.195 DEBUG:teuthology.provision.fog.smithi069:FOG request: GET /tasktype {"name": "deploy"}
2023-11-14T17:44:17.226 DEBUG:teuthology.provision.fog.smithi069:FOG request: POST /host/174/task {"taskTypeID": 1}
2023-11-14T17:44:17.232 INFO:teuthology.provision.fog.smithi196:Scheduling deploy of rhel 8.4
2023-11-14T17:44:17.232 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /task/active None
2023-11-14T17:44:17.277 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /tasktype {"name": "deploy"}
2023-11-14T17:44:17.299 DEBUG:teuthology.provision.fog.smithi069:FOG request: GET /task/active None
2023-11-14T17:44:17.315 DEBUG:teuthology.provision.fog.smithi196:FOG request: POST /host/301/task {"taskTypeID": 1}
2023-11-14T17:44:17.366 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /task/active None
2023-11-14T17:44:17.382 INFO:teuthology.orchestra.console.smithi069:Power off
2023-11-14T17:44:17.383 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-14T17:44:17.405 DEBUG:teuthology.orchestra.console.smithi069:power off output: Chassis Power Control: Down/Off
2023-11-14T17:44:17.405 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T17:44:17.427 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on
2023-11-14T17:44:17.449 INFO:teuthology.orchestra.console.smithi196:Power off
2023-11-14T17:44:17.450 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-14T17:44:17.476 DEBUG:teuthology.orchestra.console.smithi196:power off output: Chassis Power Control: Down/Off
2023-11-14T17:44:17.477 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T17:44:17.502 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on
2023-11-14T17:44:21.429 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T17:44:21.503 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T17:44:21.539 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on
2023-11-14T17:44:21.613 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on
2023-11-14T17:44:25.542 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T17:44:25.615 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T17:44:25.652 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off
2023-11-14T17:44:25.725 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on
2023-11-14T17:44:25.753 INFO:teuthology.orchestra.console.smithi069:Power off completed
2023-11-14T17:44:25.854 INFO:teuthology.orchestra.console.smithi069:Power on
2023-11-14T17:44:25.855 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-14T17:44:25.878 DEBUG:teuthology.orchestra.console.smithi069:power on output: Chassis Power Control: Up/On
2023-11-14T17:44:25.879 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T17:44:25.900 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off
2023-11-14T17:44:29.730 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T17:44:29.839 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off
2023-11-14T17:44:29.902 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T17:44:29.940 INFO:teuthology.orchestra.console.smithi196:Power off completed
2023-11-14T17:44:30.011 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off
2023-11-14T17:44:30.042 INFO:teuthology.orchestra.console.smithi196:Power on
2023-11-14T17:44:30.042 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-14T17:44:30.062 DEBUG:teuthology.orchestra.console.smithi196:power on output: Chassis Power Control: Up/On
2023-11-14T17:44:30.063 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T17:44:30.084 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off
2023-11-14T17:44:34.014 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T17:44:34.085 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T17:44:34.124 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on
2023-11-14T17:44:34.194 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off
2023-11-14T17:44:34.225 INFO:teuthology.orchestra.console.smithi069:Power on completed
2023-11-14T17:44:34.326 INFO:teuthology.provision.fog.smithi069:Waiting for deploy to finish
2023-11-14T17:44:34.326 DEBUG:teuthology.provision.fog.smithi069:FOG request: GET /task/active None
2023-11-14T17:44:38.199 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T17:44:38.308 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on
2023-11-14T17:44:38.409 INFO:teuthology.orchestra.console.smithi196:Power on completed
2023-11-14T17:44:38.509 INFO:teuthology.provision.fog.smithi196:Waiting for deploy to finish
2023-11-14T17:44:38.510 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /task/active None
2023-11-14T17:44:49.398 DEBUG:teuthology.provision.fog.smithi069:FOG request: GET /task/active None
2023-11-14T17:44:53.566 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /task/active None
2023-11-14T17:45:04.470 DEBUG:teuthology.provision.fog.smithi069:FOG request: GET /task/active None
2023-11-14T17:45:08.622 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /task/active None
2023-11-14T17:45:19.539 DEBUG:teuthology.provision.fog.smithi069:FOG request: GET /task/active None
2023-11-14T17:45:23.695 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /task/active None
2023-11-14T17:45:34.610 DEBUG:teuthology.provision.fog.smithi069:FOG request: GET /task/active None
2023-11-14T17:45:38.770 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /task/active None
2023-11-14T17:45:49.687 DEBUG:teuthology.provision.fog.smithi069:FOG request: GET /task/active None
2023-11-14T17:45:53.839 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /task/active None
2023-11-14T17:46:04.766 DEBUG:teuthology.provision.fog.smithi069:FOG request: GET /task/active None
2023-11-14T17:46:08.910 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /task/active None
2023-11-14T17:46:19.852 DEBUG:teuthology.provision.fog.smithi069:FOG request: GET /task/active None
2023-11-14T17:46:23.982 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /task/active None
2023-11-14T17:46:34.938 DEBUG:teuthology.provision.fog.smithi069:FOG request: GET /task/active None
2023-11-14T17:46:39.062 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /task/active None
2023-11-14T17:46:50.026 DEBUG:teuthology.provision.fog.smithi069:FOG request: GET /task/active None
2023-11-14T17:46:54.138 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /task/active None
2023-11-14T17:47:05.123 DEBUG:teuthology.provision.fog.smithi069:FOG request: GET /task/active None
2023-11-14T17:47:09.218 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /task/active None
2023-11-14T17:47:20.210 DEBUG:teuthology.provision.fog.smithi069:FOG request: GET /task/active None
2023-11-14T17:47:20.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:47:23.354 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69
2023-11-14T17:47:24.293 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /task/active None
2023-11-14T17:47:24.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:47:27.454 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196
2023-11-14T17:47:29.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:47:32.410 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69
2023-11-14T17:47:33.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:47:36.542 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196
2023-11-14T17:47:38.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:47:41.466 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69
2023-11-14T17:47:42.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:47:45.626 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196
2023-11-14T17:47:47.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:47:50.526 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69
2023-11-14T17:47:51.628 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:47:54.682 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196
2023-11-14T17:47:56.530 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:47:59.614 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69
2023-11-14T17:48:00.685 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:48:03.738 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196
2023-11-14T17:48:05.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:48:08.702 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69
2023-11-14T17:48:09.741 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:48:12.794 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196
2023-11-14T17:48:14.705 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:48:18.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:49:14.763 WARNING:teuthology.provision.fog.smithi069:timed out
2023-11-14T17:49:18.802 WARNING:teuthology.provision.fog.smithi196:timed out
2023-11-14T17:49:20.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:49:21.116 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-14T17:49:21.117 DEBUG:teuthology.orchestra.run.smithi069:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-14T17:49:21.415 INFO:teuthology.provision.fog.smithi069:Node is ready
2023-11-14T17:49:21.416 DEBUG:teuthology.orchestra.run.smithi069:> hostname
2023-11-14T17:49:21.430 INFO:teuthology.orchestra.run.smithi069.stdout:smithi069.front.sepia.ceph.com
2023-11-14T17:49:21.431 DEBUG:teuthology.orchestra.run.smithi069:> grep smithi069.front.sepia.ceph.com /etc/hosts
2023-11-14T17:49:21.485 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69 smithi069.front.sepia.ceph.com smithi069
2023-11-14T17:49:21.486 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname smithi069
2023-11-14T17:49:21.605 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i -e 's/smithi069.front.sepia.ceph.com/smithi069/g' /etc/hosts
2023-11-14T17:49:21.630 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i -e 's/172.21.15.69/172.21.15.69/g' /etc/hosts
2023-11-14T17:49:21.695 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i -e 's/smithi069.front.sepia.ceph.com/smithi069/g' /etc/hostname
2023-11-14T17:49:21.758 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname smithi069
2023-11-14T17:49:21.821 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release
2023-11-14T17:49:21.876 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="Red Hat Enterprise Linux"
2023-11-14T17:49:21.876 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="8.4 (Ootpa)"
2023-11-14T17:49:21.876 INFO:teuthology.orchestra.run.smithi069.stdout:ID="rhel"
2023-11-14T17:49:21.876 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="fedora"
2023-11-14T17:49:21.877 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="8.4"
2023-11-14T17:49:21.877 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el8"
2023-11-14T17:49:21.877 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-11-14T17:49:21.877 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31"
2023-11-14T17:49:21.877 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-11-14T17:49:21.878 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://www.redhat.com/"
2023-11-14T17:49:21.878 INFO:teuthology.orchestra.run.smithi069.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-14T17:49:21.878 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-14T17:49:21.879 INFO:teuthology.orchestra.run.smithi069.stdout:
2023-11-14T17:49:21.879 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-14T17:49:21.879 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-11-14T17:49:21.879 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-14T17:49:21.880 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-11-14T17:49:21.881 INFO:teuthology.provision.fog.smithi069:Deploy complete!
2023-11-14T17:49:21.883 DEBUG:teuthology.parallel:result is None
2023-11-14T17:49:24.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:49:25.138 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-14T17:49:25.138 DEBUG:teuthology.orchestra.run.smithi196:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-14T17:49:25.386 INFO:teuthology.provision.fog.smithi196:Node is ready
2023-11-14T17:49:25.386 DEBUG:teuthology.orchestra.run.smithi196:> hostname
2023-11-14T17:49:25.441 INFO:teuthology.orchestra.run.smithi196.stdout:smithi196.front.sepia.ceph.com
2023-11-14T17:49:25.441 DEBUG:teuthology.orchestra.run.smithi196:> grep smithi196.front.sepia.ceph.com /etc/hosts
2023-11-14T17:49:25.497 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.196 smithi196.front.sepia.ceph.com smithi196
2023-11-14T17:49:25.497 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname smithi196
2023-11-14T17:49:25.609 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/smithi196.front.sepia.ceph.com/smithi196/g' /etc/hosts
2023-11-14T17:49:25.636 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/172.21.15.196/172.21.15.196/g' /etc/hosts
2023-11-14T17:49:25.701 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/smithi196.front.sepia.ceph.com/smithi196/g' /etc/hostname
2023-11-14T17:49:25.766 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname smithi196
2023-11-14T17:49:25.830 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release
2023-11-14T17:49:25.885 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="Red Hat Enterprise Linux"
2023-11-14T17:49:25.885 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="8.4 (Ootpa)"
2023-11-14T17:49:25.886 INFO:teuthology.orchestra.run.smithi196.stdout:ID="rhel"
2023-11-14T17:49:25.886 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="fedora"
2023-11-14T17:49:25.886 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="8.4"
2023-11-14T17:49:25.886 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el8"
2023-11-14T17:49:25.886 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-11-14T17:49:25.887 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31"
2023-11-14T17:49:25.887 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-11-14T17:49:25.887 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://www.redhat.com/"
2023-11-14T17:49:25.887 INFO:teuthology.orchestra.run.smithi196.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-14T17:49:25.888 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-14T17:49:25.888 INFO:teuthology.orchestra.run.smithi196.stdout:
2023-11-14T17:49:25.888 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-14T17:49:25.888 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-11-14T17:49:25.888 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-14T17:49:25.889 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-11-14T17:49:25.889 INFO:teuthology.provision.fog.smithi196:Deploy complete!
2023-11-14T17:49:25.890 DEBUG:teuthology.parallel:result is None
2023-11-14T17:49:26.036 INFO:teuthology.lock.ops:Checking smithi069.front.sepia.ceph.com
2023-11-14T17:49:26.037 INFO:teuthology.lock.ops:New key found. Updating...
2023-11-14T17:49:26.065 INFO:teuthology.lock.ops:Checking smithi196.front.sepia.ceph.com
2023-11-14T17:49:26.065 INFO:teuthology.lock.ops:New key found. Updating...
2023-11-14T17:49:26.091 INFO:teuthology.lock.ops:Updating [smithi069.front.sepia.ceph.com]: set os type and version on server
2023-11-14T17:49:26.092 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com'
2023-11-14T17:49:26.092 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:49:26.159 DEBUG:teuthology.orchestra.run.smithi069:> true
2023-11-14T17:49:26.183 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com'
2023-11-14T17:49:26.184 DEBUG:teuthology.orchestra.run.smithi069:> uname -m
2023-11-14T17:49:26.239 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64
2023-11-14T17:49:26.239 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release
2023-11-14T17:49:26.295 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="Red Hat Enterprise Linux"
2023-11-14T17:49:26.296 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="8.4 (Ootpa)"
2023-11-14T17:49:26.296 INFO:teuthology.orchestra.run.smithi069.stdout:ID="rhel"
2023-11-14T17:49:26.296 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="fedora"
2023-11-14T17:49:26.296 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="8.4"
2023-11-14T17:49:26.297 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el8"
2023-11-14T17:49:26.297 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-11-14T17:49:26.297 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31"
2023-11-14T17:49:26.298 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-11-14T17:49:26.298 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://www.redhat.com/"
2023-11-14T17:49:26.298 INFO:teuthology.orchestra.run.smithi069.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-14T17:49:26.298 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-14T17:49:26.298 INFO:teuthology.orchestra.run.smithi069.stdout:
2023-11-14T17:49:26.299 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-14T17:49:26.299 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-11-14T17:49:26.299 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-14T17:49:26.299 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-11-14T17:49:26.300 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server
2023-11-14T17:49:26.315 INFO:teuthology.lock.ops:Updating [smithi196.front.sepia.ceph.com]: set os type and version on server
2023-11-14T17:49:26.315 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com'
2023-11-14T17:49:26.316 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-14T17:49:26.384 DEBUG:teuthology.orchestra.run.smithi196:> true
2023-11-14T17:49:26.452 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com'
2023-11-14T17:49:26.452 DEBUG:teuthology.orchestra.run.smithi196:> uname -m
2023-11-14T17:49:26.506 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64
2023-11-14T17:49:26.506 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release
2023-11-14T17:49:26.560 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="Red Hat Enterprise Linux"
2023-11-14T17:49:26.560 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="8.4 (Ootpa)"
2023-11-14T17:49:26.560 INFO:teuthology.orchestra.run.smithi196.stdout:ID="rhel"
2023-11-14T17:49:26.560 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="fedora"
2023-11-14T17:49:26.561 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="8.4"
2023-11-14T17:49:26.561 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el8"
2023-11-14T17:49:26.561 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-11-14T17:49:26.561 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31"
2023-11-14T17:49:26.561 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-11-14T17:49:26.562 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://www.redhat.com/"
2023-11-14T17:49:26.562 INFO:teuthology.orchestra.run.smithi196.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-14T17:49:26.562 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-14T17:49:26.562 INFO:teuthology.orchestra.run.smithi196.stdout:
2023-11-14T17:49:26.563 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-14T17:49:26.563 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-11-14T17:49:26.563 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-14T17:49:26.563 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-11-14T17:49:26.564 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server
2023-11-14T17:49:26.591 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T17:49:26.680 INFO:teuthology.dispatcher.supervisor:Running job 7457497
2023-11-14T17:49:26.681 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-11-14_15:38:06-rgw-wip-yuri2-testing-2023-11-13-0820-pacific-distro-default-smithi/7457497 --name yuriw-2023-11-14_15:38:06-rgw-wip-yuri2-testing-2023-11-13-0820-pacific-distro-default-smithi --description rgw/multisite/{clusters frontend/civetweb ignore-pg-availability omap_limits overrides realms/three-zones supported-random-distro$/{rhel_8} tasks/test_multi} -- /home/teuthworker/archive/yuriw-2023-11-14_15:38:06-rgw-wip-yuri2-testing-2023-11-13-0820-pacific-distro-default-smithi/7457497/orig.config.yaml
2023-11-14T17:49:26.684 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-11-14_15:38:06-rgw-wip-yuri2-testing-2023-11-13-0820-pacific-distro-default-smithi/7457497
2023-11-14T17:49:26.685 INFO:teuthology.dispatcher.supervisor:Job PID: 3351348
2023-11-14T17:49:26.685 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-11-14T17:51:26.745 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T17:53:26.919 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T17:55:27.083 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T17:57:27.245 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T17:59:27.419 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T18:01:27.587 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T18:03:27.761 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T18:05:27.924 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T18:07:28.087 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T18:09:28.260 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T18:11:28.419 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-14T18:11:28.484 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-11-14T18:11:28.586 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-11-14T18:11:28.586 INFO:teuthology.nuke:Checking targets against current locks
2023-11-14T18:11:28.634 DEBUG:teuthology.nuke:shortname: smithi069
2023-11-14T18:11:28.634 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-14T18:11:28.656 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-14_15:38:06-rgw-wip-yuri2-testing-2023-11-13-0820-pacific-distro-default-smithi/7457497', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-14 17:44:15.968772', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBErbO4Nocsjqcu3wSH3Eu36svBBGhE2FQpAjY6OFj0aTeKZEUlJ0LjmqwcWF3+uHhgBuZDgTm0UGKA3uJLLEOn4='}
2023-11-14T18:11:28.659 DEBUG:teuthology.nuke:shortname: smithi196
2023-11-14T18:11:28.660 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-14T18:11:28.680 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-14_15:38:06-rgw-wip-yuri2-testing-2023-11-13-0820-pacific-distro-default-smithi/7457497', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-14 17:44:15.970626', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB9ZkOzorg/u/y5XQ2LAAAc+tsLLuhXkGY13dQnRx5ddn9k+KQ0TZL1h527GXNfon2yek6grtGJDrgqbyDpkd+c='}
2023-11-14T18:11:28.709 INFO:teuthology.orchestra.console.smithi069:Power off
2023-11-14T18:11:28.710 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-14T18:11:28.733 DEBUG:teuthology.orchestra.console.smithi069:power off output: Chassis Power Control: Down/Off
2023-11-14T18:11:28.734 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T18:11:28.750 INFO:teuthology.orchestra.console.smithi196:Power off
2023-11-14T18:11:28.751 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-14T18:11:28.761 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off
2023-11-14T18:11:28.776 DEBUG:teuthology.orchestra.console.smithi196:power off output: Chassis Power Control: Down/Off
2023-11-14T18:11:28.776 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-14T18:11:28.797 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off
2023-11-14T18:11:28.861 INFO:teuthology.orchestra.console.smithi069:Power off completed
2023-11-14T18:11:28.899 INFO:teuthology.orchestra.console.smithi196:Power off completed
2023-11-14T18:11:29.013 INFO:teuthology.lock.ops:unlocked: smithi069.front.sepia.ceph.com
2023-11-14T18:11:29.037 INFO:teuthology.lock.ops:unlocked: smithi196.front.sepia.ceph.com