2023-10-05T02:56:28.628 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-10-05T02:56:28.629 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-05T02:56:28.833 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f04258a2bb0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f042638e670>, signals=[15])
2023-10-05T02:56:28.834 INFO:teuthology.lock.ops:Start node 'smithi016.front.sepia.ceph.com' reimaging
2023-10-05T02:56:28.834 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: reset os type and version on server
2023-10-05T02:56:28.834 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server
2023-10-05T02:56:28.859 INFO:teuthology.lock.ops:Start node 'smithi040.front.sepia.ceph.com' reimaging
2023-10-05T02:56:28.859 INFO:teuthology.lock.ops:Updating [smithi040.front.sepia.ceph.com]: reset os type and version on server
2023-10-05T02:56:28.860 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server
2023-10-05T02:56:28.861 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /host {"name": "smithi016"}
2023-10-05T02:56:28.885 DEBUG:teuthology.provision.fog.smithi040:FOG request: GET /host {"name": "smithi040"}
2023-10-05T02:56:28.942 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /image {"name": "smithi_rhel_8.4"}
2023-10-05T02:56:28.951 DEBUG:teuthology.provision.fog.smithi040:FOG request: GET /image {"name": "smithi_rhel_8.4"}
2023-10-05T02:56:28.984 DEBUG:teuthology.provision.fog.smithi016:Requesting image smithi_rhel_8.4 (ID 7)
2023-10-05T02:56:28.985 DEBUG:teuthology.provision.fog.smithi016:FOG request: PUT /host/122 {"imageID": 7}
2023-10-05T02:56:28.993 DEBUG:teuthology.provision.fog.smithi040:Requesting image smithi_rhel_8.4 (ID 7)
2023-10-05T02:56:28.993 DEBUG:teuthology.provision.fog.smithi040:FOG request: PUT /host/145 {"imageID": 7}
2023-10-05T02:56:29.028 INFO:teuthology.provision.fog.smithi016:Scheduling deploy of rhel 8.4
2023-10-05T02:56:29.029 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-10-05T02:56:29.042 INFO:teuthology.provision.fog.smithi040:Scheduling deploy of rhel 8.4
2023-10-05T02:56:29.042 DEBUG:teuthology.provision.fog.smithi040:FOG request: GET /task/active None
2023-10-05T02:56:29.075 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /tasktype {"name": "deploy"}
2023-10-05T02:56:29.093 DEBUG:teuthology.provision.fog.smithi040:FOG request: GET /tasktype {"name": "deploy"}
2023-10-05T02:56:29.109 DEBUG:teuthology.provision.fog.smithi016:FOG request: POST /host/122/task {"taskTypeID": 1}
2023-10-05T02:56:29.130 DEBUG:teuthology.provision.fog.smithi040:FOG request: POST /host/145/task {"taskTypeID": 1}
2023-10-05T02:56:29.178 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-10-05T02:56:29.181 DEBUG:teuthology.provision.fog.smithi040:FOG request: GET /task/active None
2023-10-05T02:56:29.260 INFO:teuthology.orchestra.console.smithi040:Power off
2023-10-05T02:56:29.261 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-10-05T02:56:29.275 INFO:teuthology.orchestra.console.smithi016:Power off
2023-10-05T02:56:29.275 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-10-05T02:56:29.289 DEBUG:teuthology.orchestra.console.smithi040:power off output: Chassis Power Control: Down/Off
2023-10-05T02:56:29.290 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-05T02:56:29.300 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off
2023-10-05T02:56:29.300 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-05T02:56:29.313 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off
2023-10-05T02:56:29.321 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off
2023-10-05T02:56:29.414 INFO:teuthology.orchestra.console.smithi040:Power off completed
2023-10-05T02:56:29.422 INFO:teuthology.orchestra.console.smithi016:Power off completed
2023-10-05T02:56:29.515 INFO:teuthology.orchestra.console.smithi040:Power on
2023-10-05T02:56:29.516 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-10-05T02:56:29.523 INFO:teuthology.orchestra.console.smithi016:Power on
2023-10-05T02:56:29.523 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-10-05T02:56:29.540 DEBUG:teuthology.orchestra.console.smithi040:power on output: Chassis Power Control: Up/On
2023-10-05T02:56:29.541 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-05T02:56:29.550 DEBUG:teuthology.orchestra.console.smithi016:power on output: Chassis Power Control: Up/On
2023-10-05T02:56:29.550 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-05T02:56:29.564 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off
2023-10-05T02:56:29.572 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off
2023-10-05T02:56:33.567 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-05T02:56:33.574 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-05T02:56:33.676 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on
2023-10-05T02:56:33.682 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on
2023-10-05T02:56:33.777 INFO:teuthology.orchestra.console.smithi040:Power on completed
2023-10-05T02:56:33.783 INFO:teuthology.orchestra.console.smithi016:Power on completed
2023-10-05T02:56:33.878 INFO:teuthology.provision.fog.smithi040:Waiting for deploy to finish
2023-10-05T02:56:33.878 DEBUG:teuthology.provision.fog.smithi040:FOG request: GET /task/active None
2023-10-05T02:56:33.884 INFO:teuthology.provision.fog.smithi016:Waiting for deploy to finish
2023-10-05T02:56:33.884 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-10-05T02:56:48.958 DEBUG:teuthology.provision.fog.smithi040:FOG request: GET /task/active None
2023-10-05T02:56:48.960 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-10-05T02:57:04.030 DEBUG:teuthology.provision.fog.smithi040:FOG request: GET /task/active None
2023-10-05T02:57:04.032 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-10-05T02:57:19.102 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-10-05T02:57:19.104 DEBUG:teuthology.provision.fog.smithi040:FOG request: GET /task/active None
2023-10-05T02:57:34.166 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-10-05T02:57:34.167 DEBUG:teuthology.provision.fog.smithi040:FOG request: GET /task/active None
2023-10-05T02:57:49.245 DEBUG:teuthology.provision.fog.smithi040:FOG request: GET /task/active None
2023-10-05T02:57:49.246 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-10-05T02:58:04.333 DEBUG:teuthology.provision.fog.smithi040:FOG request: GET /task/active None
2023-10-05T02:58:04.335 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-10-05T02:58:19.412 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-10-05T02:58:19.413 DEBUG:teuthology.provision.fog.smithi040:FOG request: GET /task/active None
2023-10-05T02:58:34.494 DEBUG:teuthology.provision.fog.smithi040:FOG request: GET /task/active None
2023-10-05T02:58:34.495 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-10-05T02:58:49.570 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-10-05T02:58:49.572 DEBUG:teuthology.provision.fog.smithi040:FOG request: GET /task/active None
2023-10-05T02:59:04.654 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-10-05T02:59:04.658 DEBUG:teuthology.provision.fog.smithi040:FOG request: GET /task/active None
2023-10-05T02:59:04.729 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-05T02:59:04.869 ERROR:teuthology.orchestra.connection:Error authenticating with smithi040.front.sepia.ceph.com: Authentication failed.
2023-10-05T02:59:04.869 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-10-05T02:59:04.870 DEBUG:teuthology.orchestra.run.smithi040:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-10-05T02:59:04.871 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-10-05T02:59:05.876 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-10-05T02:59:19.732 DEBUG:teuthology.provision.fog.smithi016:FOG request: GET /task/active None
2023-10-05T02:59:19.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-05T02:59:38.238 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16
2023-10-05T02:59:44.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-05T02:59:47.290 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16
2023-10-05T02:59:53.292 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-05T02:59:56.346 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16
2023-10-05T03:00:02.347 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-05T03:00:57.598 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2023-10-05T03:00:57.600 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi040
2023-10-05T03:01:02.352 WARNING:teuthology.provision.fog.smithi016:timed out
2023-10-05T03:01:06.603 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com'
2023-10-05T03:01:06.604 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-05T03:01:06.906 DEBUG:teuthology.orchestra.run.smithi040:> true
2023-10-05T03:01:07.171 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com'
2023-10-05T03:01:07.171 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-10-05T03:01:07.171 DEBUG:teuthology.orchestra.run.smithi040:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-10-05T03:01:07.224 INFO:teuthology.provision.fog.smithi040:Node is ready
2023-10-05T03:01:07.224 DEBUG:teuthology.orchestra.run.smithi040:> hostname
2023-10-05T03:01:07.278 INFO:teuthology.orchestra.run.smithi040.stdout:smithi040.front.sepia.ceph.com
2023-10-05T03:01:07.279 DEBUG:teuthology.orchestra.run.smithi040:> grep smithi040.front.sepia.ceph.com /etc/hosts
2023-10-05T03:01:07.333 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40 smithi040.front.sepia.ceph.com smithi040
2023-10-05T03:01:07.333 DEBUG:teuthology.orchestra.run.smithi040:> sudo hostname smithi040
2023-10-05T03:01:07.455 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i -e 's/smithi040.front.sepia.ceph.com/smithi040/g' /etc/hosts
2023-10-05T03:01:07.480 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i -e 's/172.21.15.40/172.21.15.40/g' /etc/hosts
2023-10-05T03:01:07.545 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i -e 's/smithi040.front.sepia.ceph.com/smithi040/g' /etc/hostname
2023-10-05T03:01:07.610 DEBUG:teuthology.orchestra.run.smithi040:> sudo hostname smithi040
2023-10-05T03:01:07.674 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release
2023-10-05T03:01:07.728 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="Red Hat Enterprise Linux"
2023-10-05T03:01:07.729 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="8.4 (Ootpa)"
2023-10-05T03:01:07.729 INFO:teuthology.orchestra.run.smithi040.stdout:ID="rhel"
2023-10-05T03:01:07.729 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="fedora"
2023-10-05T03:01:07.729 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="8.4"
2023-10-05T03:01:07.729 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el8"
2023-10-05T03:01:07.729 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-10-05T03:01:07.729 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31"
2023-10-05T03:01:07.729 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-10-05T03:01:07.729 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://www.redhat.com/"
2023-10-05T03:01:07.729 INFO:teuthology.orchestra.run.smithi040.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-10-05T03:01:07.730 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-10-05T03:01:07.730 INFO:teuthology.orchestra.run.smithi040.stdout:
2023-10-05T03:01:07.730 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-10-05T03:01:07.730 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-10-05T03:01:07.730 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-10-05T03:01:07.730 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-10-05T03:01:07.731 INFO:teuthology.provision.fog.smithi040:Deploy complete!
2023-10-05T03:01:07.732 DEBUG:teuthology.parallel:result is None
2023-10-05T03:01:08.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-05T03:01:08.606 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-10-05T03:01:08.606 DEBUG:teuthology.orchestra.run.smithi016:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-10-05T03:01:08.843 INFO:teuthology.provision.fog.smithi016:Node is ready
2023-10-05T03:01:08.843 DEBUG:teuthology.orchestra.run.smithi016:> hostname
2023-10-05T03:01:08.858 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016.front.sepia.ceph.com
2023-10-05T03:01:08.859 DEBUG:teuthology.orchestra.run.smithi016:> grep smithi016.front.sepia.ceph.com /etc/hosts
2023-10-05T03:01:08.915 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16 smithi016.front.sepia.ceph.com smithi016
2023-10-05T03:01:08.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname smithi016
2023-10-05T03:01:09.021 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/smithi016.front.sepia.ceph.com/smithi016/g' /etc/hosts
2023-10-05T03:01:09.047 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/172.21.15.16/172.21.15.16/g' /etc/hosts
2023-10-05T03:01:09.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/smithi016.front.sepia.ceph.com/smithi016/g' /etc/hostname
2023-10-05T03:01:09.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname smithi016
2023-10-05T03:01:09.243 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release
2023-10-05T03:01:09.298 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Red Hat Enterprise Linux"
2023-10-05T03:01:09.298 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8.4 (Ootpa)"
2023-10-05T03:01:09.298 INFO:teuthology.orchestra.run.smithi016.stdout:ID="rhel"
2023-10-05T03:01:09.298 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="fedora"
2023-10-05T03:01:09.298 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8.4"
2023-10-05T03:01:09.299 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8"
2023-10-05T03:01:09.299 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-10-05T03:01:09.299 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31"
2023-10-05T03:01:09.299 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-10-05T03:01:09.299 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.redhat.com/"
2023-10-05T03:01:09.299 INFO:teuthology.orchestra.run.smithi016.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-10-05T03:01:09.299 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-10-05T03:01:09.299 INFO:teuthology.orchestra.run.smithi016.stdout:
2023-10-05T03:01:09.299 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-10-05T03:01:09.299 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-10-05T03:01:09.300 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-10-05T03:01:09.300 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-10-05T03:01:09.300 INFO:teuthology.provision.fog.smithi016:Deploy complete!
2023-10-05T03:01:09.301 DEBUG:teuthology.parallel:result is None
2023-10-05T03:01:09.454 INFO:teuthology.lock.ops:Checking smithi016.front.sepia.ceph.com
2023-10-05T03:01:09.454 INFO:teuthology.lock.ops:New key found. Updating...
2023-10-05T03:01:09.480 INFO:teuthology.lock.ops:Checking smithi040.front.sepia.ceph.com
2023-10-05T03:01:09.480 INFO:teuthology.lock.ops:New key found. Updating...
2023-10-05T03:01:09.497 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: set os type and version on server
2023-10-05T03:01:09.498 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com'
2023-10-05T03:01:09.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-05T03:01:09.568 DEBUG:teuthology.orchestra.run.smithi016:> true
2023-10-05T03:01:09.629 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com'
2023-10-05T03:01:09.629 DEBUG:teuthology.orchestra.run.smithi016:> uname -m
2023-10-05T03:01:09.684 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64
2023-10-05T03:01:09.685 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release
2023-10-05T03:01:09.739 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Red Hat Enterprise Linux"
2023-10-05T03:01:09.739 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8.4 (Ootpa)"
2023-10-05T03:01:09.739 INFO:teuthology.orchestra.run.smithi016.stdout:ID="rhel"
2023-10-05T03:01:09.739 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="fedora"
2023-10-05T03:01:09.740 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8.4"
2023-10-05T03:01:09.740 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8"
2023-10-05T03:01:09.740 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-10-05T03:01:09.740 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31"
2023-10-05T03:01:09.740 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-10-05T03:01:09.740 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.redhat.com/"
2023-10-05T03:01:09.740 INFO:teuthology.orchestra.run.smithi016.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-10-05T03:01:09.740 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-10-05T03:01:09.740 INFO:teuthology.orchestra.run.smithi016.stdout:
2023-10-05T03:01:09.740 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-10-05T03:01:09.740 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-10-05T03:01:09.740 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-10-05T03:01:09.741 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-10-05T03:01:09.741 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server
2023-10-05T03:01:09.763 INFO:teuthology.lock.ops:Updating [smithi040.front.sepia.ceph.com]: set os type and version on server
2023-10-05T03:01:09.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com'
2023-10-05T03:01:09.764 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-10-05T03:01:09.834 DEBUG:teuthology.orchestra.run.smithi040:> true
2023-10-05T03:01:09.903 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com'
2023-10-05T03:01:09.904 DEBUG:teuthology.orchestra.run.smithi040:> uname -m
2023-10-05T03:01:09.960 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64
2023-10-05T03:01:09.961 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release
2023-10-05T03:01:10.017 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="Red Hat Enterprise Linux"
2023-10-05T03:01:10.018 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="8.4 (Ootpa)"
2023-10-05T03:01:10.018 INFO:teuthology.orchestra.run.smithi040.stdout:ID="rhel"
2023-10-05T03:01:10.018 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="fedora"
2023-10-05T03:01:10.018 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="8.4"
2023-10-05T03:01:10.018 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el8"
2023-10-05T03:01:10.018 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-10-05T03:01:10.018 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31"
2023-10-05T03:01:10.018 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-10-05T03:01:10.018 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://www.redhat.com/"
2023-10-05T03:01:10.018 INFO:teuthology.orchestra.run.smithi040.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-10-05T03:01:10.018 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-10-05T03:01:10.019 INFO:teuthology.orchestra.run.smithi040.stdout:
2023-10-05T03:01:10.019 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-10-05T03:01:10.019 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-10-05T03:01:10.019 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-10-05T03:01:10.019 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-10-05T03:01:10.019 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server
2023-10-05T03:01:10.038 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-05T03:01:10.132 INFO:teuthology.dispatcher.supervisor:Running job 7410818
2023-10-05T03:01:10.132 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-10-04_19:55:00-fs-wip-yuri-testing-2023-10-04-0859-quincy-distro-default-smithi/7410818 --name yuriw-2023-10-04_19:55:00-fs-wip-yuri-testing-2023-10-04-0859-quincy-distro-default-smithi --description fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} -- /home/teuthworker/archive/yuriw-2023-10-04_19:55:00-fs-wip-yuri-testing-2023-10-04-0859-quincy-distro-default-smithi/7410818/orig.config.yaml
2023-10-05T03:01:10.135 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-10-04_19:55:00-fs-wip-yuri-testing-2023-10-04-0859-quincy-distro-default-smithi/7410818
2023-10-05T03:01:10.135 INFO:teuthology.dispatcher.supervisor:Job PID: 4143758
2023-10-05T03:01:10.135 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-10-05T03:03:10.235 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-05T03:05:10.407 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-05T03:07:10.575 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-05T03:09:10.733 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-05T03:11:10.908 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-05T03:13:11.071 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-05T03:15:11.239 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-05T03:17:11.458 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-05T03:19:11.643 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-05T03:21:11.808 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-05T03:23:11.947 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-05T03:25:12.115 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-05T03:27:12.261 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-10-05T03:27:12.323 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2023-10-05T03:27:12.440 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-10-05T03:27:12.440 INFO:teuthology.nuke:Checking targets against current locks
2023-10-05T03:27:12.488 DEBUG:teuthology.nuke:shortname: smithi016
2023-10-05T03:27:12.489 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-10-05T03:27:12.510 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-04_19:55:00-fs-wip-yuri-testing-2023-10-04-0859-quincy-distro-default-smithi/7410818', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-05 02:56:27.384075', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM+MdpFYOwZxrKDSOFpBT+Z1e/R70QePAt9DmTfVEsvRtX2ZIEoxTNWKJ/MQs1sdBCvQi5yV2hNJgqnGBsf70BI='}
2023-10-05T03:27:12.514 DEBUG:teuthology.nuke:shortname: smithi040
2023-10-05T03:27:12.514 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-10-05T03:27:12.535 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-04_19:55:00-fs-wip-yuri-testing-2023-10-04-0859-quincy-distro-default-smithi/7410818', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-05 02:56:27.385440', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJrtlctISCs+KRnOq5lj5M5RuoXIaIXTk1qX9aQqApJYCCgy8hTVUg2Mk8d15XlRGnV0mTp4r8mAFb2MOS1VbOQ='}
2023-10-05T03:27:12.558 INFO:teuthology.orchestra.console.smithi016:Power off
2023-10-05T03:27:12.559 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-10-05T03:27:12.578 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off
2023-10-05T03:27:12.579 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-05T03:27:12.594 INFO:teuthology.orchestra.console.smithi040:Power off
2023-10-05T03:27:12.594 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-10-05T03:27:12.602 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off
2023-10-05T03:27:12.614 DEBUG:teuthology.orchestra.console.smithi040:power off output: Chassis Power Control: Down/Off
2023-10-05T03:27:12.615 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-10-05T03:27:12.635 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off
2023-10-05T03:27:12.703 INFO:teuthology.orchestra.console.smithi016:Power off completed
2023-10-05T03:27:12.736 INFO:teuthology.orchestra.console.smithi040:Power off completed
2023-10-05T03:27:12.863 INFO:teuthology.lock.ops:unlocked: smithi016.front.sepia.ceph.com
2023-10-05T03:27:12.888 INFO:teuthology.lock.ops:unlocked: smithi040.front.sepia.ceph.com