2023-11-09T22:16:07.534 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2
2023-11-09T22:16:07.535 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T22:16:07.667 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f46013dfaf0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f4601eca670>, signals=[15])
2023-11-09T22:16:07.668 INFO:teuthology.lock.ops:Start node 'smithi096.front.sepia.ceph.com' reimaging
2023-11-09T22:16:07.668 INFO:teuthology.lock.ops:Updating [smithi096.front.sepia.ceph.com]: reset os type and version on server
2023-11-09T22:16:07.669 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server
2023-11-09T22:16:07.691 INFO:teuthology.lock.ops:Start node 'smithi204.front.sepia.ceph.com' reimaging
2023-11-09T22:16:07.691 INFO:teuthology.lock.ops:Updating [smithi204.front.sepia.ceph.com]: reset os type and version on server
2023-11-09T22:16:07.692 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server
2023-11-09T22:16:07.693 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /host {"name": "smithi096"}
2023-11-09T22:16:07.716 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /host {"name": "smithi204"}
2023-11-09T22:16:07.800 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /image {"name": "smithi_rhel_8.4"}
2023-11-09T22:16:07.813 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /image {"name": "smithi_rhel_8.4"}
2023-11-09T22:16:07.847 DEBUG:teuthology.provision.fog.smithi204:Requesting image smithi_rhel_8.4 (ID 7)
2023-11-09T22:16:07.847 DEBUG:teuthology.provision.fog.smithi204:FOG request: PUT /host/308 {"imageID": 7}
2023-11-09T22:16:07.860 DEBUG:teuthology.provision.fog.smithi096:Requesting image smithi_rhel_8.4 (ID 7)
2023-11-09T22:16:07.861 DEBUG:teuthology.provision.fog.smithi096:FOG request: PUT /host/201 {"imageID": 7}
2023-11-09T22:16:07.906 INFO:teuthology.provision.fog.smithi204:Scheduling deploy of rhel 8.4
2023-11-09T22:16:07.906 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None
2023-11-09T22:16:07.919 INFO:teuthology.provision.fog.smithi096:Scheduling deploy of rhel 8.4
2023-11-09T22:16:07.920 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None
2023-11-09T22:16:08.038 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /tasktype {"name": "deploy"}
2023-11-09T22:16:08.044 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /tasktype {"name": "deploy"}
2023-11-09T22:16:08.083 DEBUG:teuthology.provision.fog.smithi096:FOG request: POST /host/201/task {"taskTypeID": 1}
2023-11-09T22:16:08.087 DEBUG:teuthology.provision.fog.smithi204:FOG request: POST /host/308/task {"taskTypeID": 1}
2023-11-09T22:16:08.146 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None
2023-11-09T22:16:08.157 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None
2023-11-09T22:16:08.296 INFO:teuthology.orchestra.console.smithi096:Power off
2023-11-09T22:16:08.297 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-09T22:16:08.310 INFO:teuthology.orchestra.console.smithi204:Power off
2023-11-09T22:16:08.311 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-09T22:16:08.319 DEBUG:teuthology.orchestra.console.smithi096:power off output: Chassis Power Control: Down/Off
2023-11-09T22:16:08.319 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T22:16:08.331 DEBUG:teuthology.orchestra.console.smithi204:power off output: Chassis Power Control: Down/Off
2023-11-09T22:16:08.331 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T22:16:08.343 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on
2023-11-09T22:16:08.355 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on
2023-11-09T22:16:12.346 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T22:16:12.356 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T22:16:12.456 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on
2023-11-09T22:16:12.465 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on
2023-11-09T22:16:16.458 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T22:16:16.466 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T22:16:16.567 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on
2023-11-09T22:16:16.574 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on
2023-11-09T22:16:20.570 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T22:16:20.577 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T22:16:20.679 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is off
2023-11-09T22:16:20.685 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off
2023-11-09T22:16:20.781 INFO:teuthology.orchestra.console.smithi096:Power off completed
2023-11-09T22:16:20.787 INFO:teuthology.orchestra.console.smithi204:Power off completed
2023-11-09T22:16:20.882 INFO:teuthology.orchestra.console.smithi096:Power on
2023-11-09T22:16:20.883 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-09T22:16:20.891 INFO:teuthology.orchestra.console.smithi204:Power on
2023-11-09T22:16:20.892 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-09T22:16:20.903 DEBUG:teuthology.orchestra.console.smithi096:power on output: Chassis Power Control: Up/On
2023-11-09T22:16:20.904 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T22:16:20.913 DEBUG:teuthology.orchestra.console.smithi204:power on output: Chassis Power Control: Up/On
2023-11-09T22:16:20.913 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T22:16:20.925 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is off
2023-11-09T22:16:20.933 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off
2023-11-09T22:16:24.929 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T22:16:24.938 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T22:16:25.040 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on
2023-11-09T22:16:25.046 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on
2023-11-09T22:16:25.141 INFO:teuthology.orchestra.console.smithi096:Power on completed
2023-11-09T22:16:25.147 INFO:teuthology.orchestra.console.smithi204:Power on completed
2023-11-09T22:16:25.242 INFO:teuthology.provision.fog.smithi096:Waiting for deploy to finish
2023-11-09T22:16:25.242 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None
2023-11-09T22:16:25.248 INFO:teuthology.provision.fog.smithi204:Waiting for deploy to finish
2023-11-09T22:16:25.249 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None
2023-11-09T22:16:40.370 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None
2023-11-09T22:16:40.371 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None
2023-11-09T22:16:55.502 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None
2023-11-09T22:16:55.503 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None
2023-11-09T22:17:10.625 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None
2023-11-09T22:17:10.627 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None
2023-11-09T22:17:25.754 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None
2023-11-09T22:17:25.756 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None
2023-11-09T22:17:40.898 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None
2023-11-09T22:17:40.900 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None
2023-11-09T22:17:56.010 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None
2023-11-09T22:17:56.011 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None
2023-11-09T22:18:11.131 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None
2023-11-09T22:18:11.133 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None
2023-11-09T22:18:26.277 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None
2023-11-09T22:18:26.279 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None
2023-11-09T22:18:41.410 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None
2023-11-09T22:18:41.412 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None
2023-11-09T22:18:56.554 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None
2023-11-09T22:18:56.556 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None
2023-11-09T22:19:11.675 DEBUG:teuthology.provision.fog.smithi096:FOG request: GET /task/active None
2023-11-09T22:19:11.676 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None
2023-11-09T22:19:11.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:19:26.814 DEBUG:teuthology.provision.fog.smithi204:FOG request: GET /task/active None
2023-11-09T22:19:26.916 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:19:29.978 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204
2023-11-09T22:19:30.298 WARNING:teuthology.provision.fog.smithi096:[Errno None] Unable to connect to port 22 on 172.21.15.96
2023-11-09T22:19:35.986 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:19:36.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:19:39.070 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204
2023-11-09T22:19:39.358 WARNING:teuthology.provision.fog.smithi096:[Errno None] Unable to connect to port 22 on 172.21.15.96
2023-11-09T22:19:45.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:19:45.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:19:48.159 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204
2023-11-09T22:19:48.410 WARNING:teuthology.provision.fog.smithi096:[Errno None] Unable to connect to port 22 on 172.21.15.96
2023-11-09T22:19:54.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:19:54.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:19:57.246 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204
2023-11-09T22:19:57.466 WARNING:teuthology.provision.fog.smithi096:[Errno None] Unable to connect to port 22 on 172.21.15.96
2023-11-09T22:20:03.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:20:03.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:20:06.330 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204
2023-11-09T22:20:12.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:20:15.386 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204
2023-11-09T22:20:21.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:20:39.930 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204
2023-11-09T22:20:45.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:20:49.018 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204
2023-11-09T22:20:55.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:20:58.074 WARNING:teuthology.provision.fog.smithi204:[Errno None] Unable to connect to port 22 on 172.21.15.204
2023-11-09T22:21:03.474 WARNING:teuthology.provision.fog.smithi096:timed out
2023-11-09T22:21:04.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:21:04.939 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-09T22:21:04.940 DEBUG:teuthology.orchestra.run.smithi204:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-09T22:21:09.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:21:09.807 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-09T22:21:09.808 DEBUG:teuthology.orchestra.run.smithi096:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-09T22:21:10.075 INFO:teuthology.provision.fog.smithi096:Node is ready
2023-11-09T22:21:10.075 DEBUG:teuthology.orchestra.run.smithi096:> hostname
2023-11-09T22:21:10.130 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096.front.sepia.ceph.com
2023-11-09T22:21:10.131 DEBUG:teuthology.orchestra.run.smithi096:> grep smithi096.front.sepia.ceph.com /etc/hosts
2023-11-09T22:21:10.186 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96 smithi096.front.sepia.ceph.com smithi096
2023-11-09T22:21:10.186 DEBUG:teuthology.orchestra.run.smithi096:> sudo hostname smithi096
2023-11-09T22:21:10.288 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i -e 's/smithi096.front.sepia.ceph.com/smithi096/g' /etc/hosts
2023-11-09T22:21:10.315 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i -e 's/172.21.15.96/172.21.15.96/g' /etc/hosts
2023-11-09T22:21:10.381 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i -e 's/smithi096.front.sepia.ceph.com/smithi096/g' /etc/hostname
2023-11-09T22:21:10.446 DEBUG:teuthology.orchestra.run.smithi096:> sudo hostname smithi096
2023-11-09T22:21:10.510 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release
2023-11-09T22:21:10.564 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="Red Hat Enterprise Linux"
2023-11-09T22:21:10.564 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="8.4 (Ootpa)"
2023-11-09T22:21:10.564 INFO:teuthology.orchestra.run.smithi096.stdout:ID="rhel"
2023-11-09T22:21:10.565 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="fedora"
2023-11-09T22:21:10.565 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="8.4"
2023-11-09T22:21:10.565 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el8"
2023-11-09T22:21:10.565 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-11-09T22:21:10.565 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31"
2023-11-09T22:21:10.566 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-11-09T22:21:10.566 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://www.redhat.com/"
2023-11-09T22:21:10.566 INFO:teuthology.orchestra.run.smithi096.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-09T22:21:10.567 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-09T22:21:10.567 INFO:teuthology.orchestra.run.smithi096.stdout:
2023-11-09T22:21:10.567 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-09T22:21:10.567 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-11-09T22:21:10.568 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-09T22:21:10.568 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-11-09T22:21:10.569 INFO:teuthology.provision.fog.smithi096:Deploy complete!
2023-11-09T22:21:10.571 DEBUG:teuthology.parallel:result is None
2023-11-09T22:21:25.219 INFO:teuthology.provision.fog.smithi204:Node is ready
2023-11-09T22:21:25.220 DEBUG:teuthology.orchestra.run.smithi204:> hostname
2023-11-09T22:21:25.276 INFO:teuthology.orchestra.run.smithi204.stdout:smithi204.front.sepia.ceph.com
2023-11-09T22:21:25.276 DEBUG:teuthology.orchestra.run.smithi204:> grep smithi204.front.sepia.ceph.com /etc/hosts
2023-11-09T22:21:25.331 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.15.204 smithi204.front.sepia.ceph.com smithi204
2023-11-09T22:21:25.331 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname smithi204
2023-11-09T22:21:25.480 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i -e 's/smithi204.front.sepia.ceph.com/smithi204/g' /etc/hosts
2023-11-09T22:21:25.505 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i -e 's/172.21.15.204/172.21.15.204/g' /etc/hosts
2023-11-09T22:21:25.569 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i -e 's/smithi204.front.sepia.ceph.com/smithi204/g' /etc/hostname
2023-11-09T22:21:25.633 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname smithi204
2023-11-09T22:21:25.697 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release
2023-11-09T22:21:25.751 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="Red Hat Enterprise Linux"
2023-11-09T22:21:25.751 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="8.4 (Ootpa)"
2023-11-09T22:21:25.752 INFO:teuthology.orchestra.run.smithi204.stdout:ID="rhel"
2023-11-09T22:21:25.752 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="fedora"
2023-11-09T22:21:25.752 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="8.4"
2023-11-09T22:21:25.752 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el8"
2023-11-09T22:21:25.753 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-11-09T22:21:25.753 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31"
2023-11-09T22:21:25.753 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-11-09T22:21:25.753 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://www.redhat.com/"
2023-11-09T22:21:25.753 INFO:teuthology.orchestra.run.smithi204.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-09T22:21:25.753 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-09T22:21:25.754 INFO:teuthology.orchestra.run.smithi204.stdout:
2023-11-09T22:21:25.754 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-09T22:21:25.754 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-11-09T22:21:25.754 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-09T22:21:25.755 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-11-09T22:21:25.755 INFO:teuthology.provision.fog.smithi204:Deploy complete!
2023-11-09T22:21:25.756 DEBUG:teuthology.parallel:result is None
2023-11-09T22:21:25.911 INFO:teuthology.lock.ops:Checking smithi096.front.sepia.ceph.com
2023-11-09T22:21:25.912 INFO:teuthology.lock.ops:New key found. Updating...
2023-11-09T22:21:25.933 INFO:teuthology.lock.ops:Checking smithi204.front.sepia.ceph.com
2023-11-09T22:21:25.934 INFO:teuthology.lock.ops:New key found. Updating...
2023-11-09T22:21:25.960 INFO:teuthology.lock.ops:Updating [smithi096.front.sepia.ceph.com]: set os type and version on server
2023-11-09T22:21:25.961 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com'
2023-11-09T22:21:25.961 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:21:26.028 DEBUG:teuthology.orchestra.run.smithi096:> true
2023-11-09T22:21:26.150 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com'
2023-11-09T22:21:26.150 DEBUG:teuthology.orchestra.run.smithi096:> uname -m
2023-11-09T22:21:26.205 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64
2023-11-09T22:21:26.205 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release
2023-11-09T22:21:26.260 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="Red Hat Enterprise Linux"
2023-11-09T22:21:26.260 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="8.4 (Ootpa)"
2023-11-09T22:21:26.260 INFO:teuthology.orchestra.run.smithi096.stdout:ID="rhel"
2023-11-09T22:21:26.261 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="fedora"
2023-11-09T22:21:26.261 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="8.4"
2023-11-09T22:21:26.261 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el8"
2023-11-09T22:21:26.261 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-11-09T22:21:26.261 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31"
2023-11-09T22:21:26.262 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-11-09T22:21:26.262 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://www.redhat.com/"
2023-11-09T22:21:26.262 INFO:teuthology.orchestra.run.smithi096.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-09T22:21:26.262 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-09T22:21:26.262 INFO:teuthology.orchestra.run.smithi096.stdout:
2023-11-09T22:21:26.263 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-09T22:21:26.263 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-11-09T22:21:26.263 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-09T22:21:26.263 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-11-09T22:21:26.264 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server
2023-11-09T22:21:26.284 INFO:teuthology.lock.ops:Updating [smithi204.front.sepia.ceph.com]: set os type and version on server
2023-11-09T22:21:26.285 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com'
2023-11-09T22:21:26.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-09T22:21:26.351 DEBUG:teuthology.orchestra.run.smithi204:> true
2023-11-09T22:21:26.373 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com'
2023-11-09T22:21:26.374 DEBUG:teuthology.orchestra.run.smithi204:> uname -m
2023-11-09T22:21:26.427 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64
2023-11-09T22:21:26.428 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release
2023-11-09T22:21:26.482 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="Red Hat Enterprise Linux"
2023-11-09T22:21:26.482 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="8.4 (Ootpa)"
2023-11-09T22:21:26.482 INFO:teuthology.orchestra.run.smithi204.stdout:ID="rhel"
2023-11-09T22:21:26.483 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="fedora"
2023-11-09T22:21:26.483 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="8.4"
2023-11-09T22:21:26.483 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el8"
2023-11-09T22:21:26.483 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-11-09T22:21:26.483 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31"
2023-11-09T22:21:26.484 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-11-09T22:21:26.484 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://www.redhat.com/"
2023-11-09T22:21:26.484 INFO:teuthology.orchestra.run.smithi204.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-09T22:21:26.484 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-09T22:21:26.485 INFO:teuthology.orchestra.run.smithi204.stdout:
2023-11-09T22:21:26.485 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-09T22:21:26.485 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-11-09T22:21:26.485 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-09T22:21:26.485 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-11-09T22:21:26.486 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server
2023-11-09T22:21:26.517 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T22:21:26.608 INFO:teuthology.dispatcher.supervisor:Running job 7453203
2023-11-09T22:21:26.609 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-11-09_21:43:29-fs-wip-yuri4-testing-2023-11-09-0758-pacific-distro-default-smithi/7453203 --name yuriw-2023-11-09_21:43:29-fs-wip-yuri4-testing-2023-11-09-0758-pacific-distro-default-smithi --description fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} -- /home/teuthworker/archive/yuriw-2023-11-09_21:43:29-fs-wip-yuri4-testing-2023-11-09-0758-pacific-distro-default-smithi/7453203/orig.config.yaml
2023-11-09T22:21:26.615 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-11-09_21:43:29-fs-wip-yuri4-testing-2023-11-09-0758-pacific-distro-default-smithi/7453203
2023-11-09T22:21:26.615 INFO:teuthology.dispatcher.supervisor:Job PID: 3694132
2023-11-09T22:21:26.615 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-11-09T22:23:26.715 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T22:25:26.887 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T22:27:27.055 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T22:29:27.219 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T22:31:27.387 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T22:33:27.567 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T22:35:27.736 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T22:37:27.911 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T22:39:28.072 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T22:41:28.239 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-09T22:41:28.304 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-11-09T22:41:28.402 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-11-09T22:41:28.403 INFO:teuthology.nuke:Checking targets against current locks
2023-11-09T22:41:28.450 DEBUG:teuthology.nuke:shortname: smithi096
2023-11-09T22:41:28.451 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-09T22:41:28.473 DEBUG:teuthology.nuke:shortname: smithi204
2023-11-09T22:41:28.473 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-09T22:41:28.476 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-09_21:43:29-fs-wip-yuri4-testing-2023-11-09-0758-pacific-distro-default-smithi/7453203', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-09 22:16:06.712139', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBANh8RkFUD2bkgf78Vje0hmfnPG+jlqagcbO3CjaPEsSD4OEcdipKLiVCn/UJWgeMhri1eHk0TigGBm1yyvM45I='}
2023-11-09T22:41:28.495 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-09_21:43:29-fs-wip-yuri4-testing-2023-11-09-0758-pacific-distro-default-smithi/7453203', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-09 22:16:06.710501', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEb6gf+//WOkK4Y7IOvGntb8Vu3/aBg5Jy02IQtxBB4kLQzKm7ak5bsUfX7AY3M+od+c4pppxeLrHUigfYLd43s='}
2023-11-09T22:41:28.522 INFO:teuthology.orchestra.console.smithi096:Power off
2023-11-09T22:41:28.523 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-09T22:41:28.544 DEBUG:teuthology.orchestra.console.smithi096:power off output: Chassis Power Control: Down/Off
2023-11-09T22:41:28.545 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T22:41:28.557 INFO:teuthology.orchestra.console.smithi204:Power off
2023-11-09T22:41:28.558 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-09T22:41:28.566 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is off
2023-11-09T22:41:28.579 DEBUG:teuthology.orchestra.console.smithi204:power off output: Chassis Power Control: Down/Off
2023-11-09T22:41:28.579 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-09T22:41:28.601 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off
2023-11-09T22:41:28.667 INFO:teuthology.orchestra.console.smithi096:Power off completed
2023-11-09T22:41:28.703 INFO:teuthology.orchestra.console.smithi204:Power off completed
2023-11-09T22:41:28.820 INFO:teuthology.lock.ops:unlocked: smithi096.front.sepia.ceph.com
2023-11-09T22:41:28.854 INFO:teuthology.lock.ops:unlocked: smithi204.front.sepia.ceph.com