2023-12-19T15:18:09.716 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e
2023-12-19T15:18:09.716 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T15:18:09.852 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f2ecbd572b0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f2eccc9e940>, signals=[15])
2023-12-19T15:18:09.853 INFO:teuthology.lock.ops:Start node 'smithi119.front.sepia.ceph.com' reimaging
2023-12-19T15:18:09.853 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: reset os type and version on server
2023-12-19T15:18:09.853 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server
2023-12-19T15:18:09.874 INFO:teuthology.lock.ops:Start node 'smithi143.front.sepia.ceph.com' reimaging
2023-12-19T15:18:09.875 INFO:teuthology.lock.ops:Updating [smithi143.front.sepia.ceph.com]: reset os type and version on server
2023-12-19T15:18:09.875 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server
2023-12-19T15:18:09.992 DEBUG:teuthology.provision.fog.smithi119:Requesting image smithi_rhel_8.4 (ID 7)
2023-12-19T15:18:10.022 DEBUG:teuthology.provision.fog.smithi143:Requesting image smithi_rhel_8.4 (ID 7)
2023-12-19T15:18:10.046 INFO:teuthology.provision.fog.smithi119:Scheduling deploy of rhel 8.4
2023-12-19T15:18:10.071 INFO:teuthology.provision.fog.smithi143:Scheduling deploy of rhel 8.4
2023-12-19T15:18:10.345 INFO:teuthology.orchestra.console.smithi119:Power off
2023-12-19T15:18:10.346 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-19T15:18:10.360 INFO:teuthology.orchestra.console.smithi143:Power off
2023-12-19T15:18:10.361 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-19T15:18:10.370 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off
2023-12-19T15:18:10.370 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T15:18:10.383 DEBUG:teuthology.orchestra.console.smithi143:power off output: Chassis Power Control: Down/Off
2023-12-19T15:18:10.383 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T15:18:10.392 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off
2023-12-19T15:18:10.406 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is off
2023-12-19T15:18:10.493 INFO:teuthology.orchestra.console.smithi119:Power off completed
2023-12-19T15:18:10.507 INFO:teuthology.orchestra.console.smithi143:Power off completed
2023-12-19T15:18:10.594 INFO:teuthology.orchestra.console.smithi119:Power on
2023-12-19T15:18:10.595 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-19T15:18:10.608 INFO:teuthology.orchestra.console.smithi143:Power on
2023-12-19T15:18:10.609 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-19T15:18:10.619 DEBUG:teuthology.orchestra.console.smithi119:power on output: Chassis Power Control: Up/On
2023-12-19T15:18:10.619 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T15:18:10.638 DEBUG:teuthology.orchestra.console.smithi143:power on output: Chassis Power Control: Up/On
2023-12-19T15:18:10.639 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T15:18:10.651 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off
2023-12-19T15:18:10.667 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is off
2023-12-19T15:18:14.653 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T15:18:14.668 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T15:18:14.764 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on
2023-12-19T15:18:14.778 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is off
2023-12-19T15:18:14.865 INFO:teuthology.orchestra.console.smithi119:Power on completed
2023-12-19T15:18:14.966 INFO:teuthology.provision.fog.smithi119:Waiting for deploy to finish
2023-12-19T15:18:18.781 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T15:18:18.890 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on
2023-12-19T15:18:18.991 INFO:teuthology.orchestra.console.smithi143:Power on completed
2023-12-19T15:18:19.091 INFO:teuthology.provision.fog.smithi143:Waiting for deploy to finish
2023-12-19T15:21:01.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:21:04.210 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119
2023-12-19T15:21:05.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:21:08.310 WARNING:teuthology.provision.fog.smithi143:[Errno None] Unable to connect to port 22 on 172.21.15.143
2023-12-19T15:21:10.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:21:13.265 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119
2023-12-19T15:21:14.314 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:21:17.397 WARNING:teuthology.provision.fog.smithi143:[Errno None] Unable to connect to port 22 on 172.21.15.143
2023-12-19T15:21:19.269 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:21:22.321 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119
2023-12-19T15:21:23.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:21:26.481 WARNING:teuthology.provision.fog.smithi143:[Errno None] Unable to connect to port 22 on 172.21.15.143
2023-12-19T15:21:28.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:21:31.377 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119
2023-12-19T15:21:32.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:21:35.537 WARNING:teuthology.provision.fog.smithi143:[Errno None] Unable to connect to port 22 on 172.21.15.143
2023-12-19T15:21:37.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:21:40.437 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119
2023-12-19T15:21:41.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:21:44.593 WARNING:teuthology.provision.fog.smithi143:[Errno None] Unable to connect to port 22 on 172.21.15.143
2023-12-19T15:21:46.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:21:49.525 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119
2023-12-19T15:21:50.595 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:21:53.649 WARNING:teuthology.provision.fog.smithi143:[Errno None] Unable to connect to port 22 on 172.21.15.143
2023-12-19T15:21:55.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:21:59.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:22:18.001 WARNING:teuthology.provision.fog.smithi143:[Errno None] Unable to connect to port 22 on 172.21.15.143
2023-12-19T15:22:24.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:22:27.061 WARNING:teuthology.provision.fog.smithi143:[Errno None] Unable to connect to port 22 on 172.21.15.143
2023-12-19T15:22:33.068 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:22:36.146 WARNING:teuthology.provision.fog.smithi143:[Errno None] Unable to connect to port 22 on 172.21.15.143
2023-12-19T15:22:42.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:22:42.619 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-19T15:22:42.620 DEBUG:teuthology.orchestra.run.smithi143:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-19T15:22:55.531 WARNING:teuthology.provision.fog.smithi119:timed out
2023-12-19T15:23:01.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:23:01.861 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-19T15:23:01.861 DEBUG:teuthology.orchestra.run.smithi119:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-19T15:23:02.114 INFO:teuthology.provision.fog.smithi119:Node is ready
2023-12-19T15:23:02.115 DEBUG:teuthology.orchestra.run.smithi119:> hostname
2023-12-19T15:23:02.167 INFO:teuthology.orchestra.run.smithi119.stdout:smithi119.front.sepia.ceph.com
2023-12-19T15:23:02.168 DEBUG:teuthology.orchestra.run.smithi119:> grep smithi119.front.sepia.ceph.com /etc/hosts
2023-12-19T15:23:02.223 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119 smithi119.front.sepia.ceph.com smithi119
2023-12-19T15:23:02.223 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname smithi119
2023-12-19T15:23:02.330 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i -e 's/smithi119.front.sepia.ceph.com/smithi119/g' /etc/hosts
2023-12-19T15:23:02.357 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i -e 's/172.21.15.119/172.21.15.119/g' /etc/hosts
2023-12-19T15:23:02.420 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i -e 's/smithi119.front.sepia.ceph.com/smithi119/g' /etc/hostname
2023-12-19T15:23:02.484 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname smithi119
2023-12-19T15:23:02.548 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release
2023-12-19T15:23:02.603 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Red Hat Enterprise Linux"
2023-12-19T15:23:02.603 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="8.4 (Ootpa)"
2023-12-19T15:23:02.603 INFO:teuthology.orchestra.run.smithi119.stdout:ID="rhel"
2023-12-19T15:23:02.603 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="fedora"
2023-12-19T15:23:02.603 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="8.4"
2023-12-19T15:23:02.603 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el8"
2023-12-19T15:23:02.603 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-12-19T15:23:02.603 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31"
2023-12-19T15:23:02.603 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-12-19T15:23:02.603 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.redhat.com/"
2023-12-19T15:23:02.603 INFO:teuthology.orchestra.run.smithi119.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-12-19T15:23:02.603 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-19T15:23:02.604 INFO:teuthology.orchestra.run.smithi119.stdout:
2023-12-19T15:23:02.604 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-19T15:23:02.604 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-12-19T15:23:02.604 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-12-19T15:23:02.604 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-12-19T15:23:02.604 INFO:teuthology.provision.fog.smithi119:Deploy complete!
2023-12-19T15:23:02.606 DEBUG:teuthology.parallel:result is None
2023-12-19T15:23:02.858 INFO:teuthology.provision.fog.smithi143:Node is ready
2023-12-19T15:23:02.858 DEBUG:teuthology.orchestra.run.smithi143:> hostname
2023-12-19T15:23:02.874 INFO:teuthology.orchestra.run.smithi143.stdout:smithi143.front.sepia.ceph.com
2023-12-19T15:23:02.874 DEBUG:teuthology.orchestra.run.smithi143:> grep smithi143.front.sepia.ceph.com /etc/hosts
2023-12-19T15:23:02.929 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143 smithi143.front.sepia.ceph.com smithi143
2023-12-19T15:23:02.929 DEBUG:teuthology.orchestra.run.smithi143:> sudo hostname smithi143
2023-12-19T15:23:03.079 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i -e 's/smithi143.front.sepia.ceph.com/smithi143/g' /etc/hosts
2023-12-19T15:23:03.103 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i -e 's/172.21.15.143/172.21.15.143/g' /etc/hosts
2023-12-19T15:23:03.168 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i -e 's/smithi143.front.sepia.ceph.com/smithi143/g' /etc/hostname
2023-12-19T15:23:03.229 DEBUG:teuthology.orchestra.run.smithi143:> sudo hostname smithi143
2023-12-19T15:23:03.298 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release
2023-12-19T15:23:03.349 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="Red Hat Enterprise Linux"
2023-12-19T15:23:03.349 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="8.4 (Ootpa)"
2023-12-19T15:23:03.349 INFO:teuthology.orchestra.run.smithi143.stdout:ID="rhel"
2023-12-19T15:23:03.349 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE="fedora"
2023-12-19T15:23:03.349 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="8.4"
2023-12-19T15:23:03.349 INFO:teuthology.orchestra.run.smithi143.stdout:PLATFORM_ID="platform:el8"
2023-12-19T15:23:03.349 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-12-19T15:23:03.349 INFO:teuthology.orchestra.run.smithi143.stdout:ANSI_COLOR="0;31"
2023-12-19T15:23:03.349 INFO:teuthology.orchestra.run.smithi143.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-12-19T15:23:03.350 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://www.redhat.com/"
2023-12-19T15:23:03.350 INFO:teuthology.orchestra.run.smithi143.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-12-19T15:23:03.350 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-19T15:23:03.350 INFO:teuthology.orchestra.run.smithi143.stdout:
2023-12-19T15:23:03.350 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-19T15:23:03.350 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-12-19T15:23:03.350 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-12-19T15:23:03.350 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-12-19T15:23:03.350 INFO:teuthology.provision.fog.smithi143:Deploy complete!
2023-12-19T15:23:03.350 DEBUG:teuthology.parallel:result is None
2023-12-19T15:23:03.503 INFO:teuthology.lock.ops:Checking smithi119.front.sepia.ceph.com
2023-12-19T15:23:03.503 INFO:teuthology.lock.ops:New key found. Updating...
2023-12-19T15:23:03.526 INFO:teuthology.lock.ops:Checking smithi143.front.sepia.ceph.com
2023-12-19T15:23:03.526 INFO:teuthology.lock.ops:New key found. Updating...
2023-12-19T15:23:03.543 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: set os type and version on server
2023-12-19T15:23:03.543 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com'
2023-12-19T15:23:03.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:23:03.610 DEBUG:teuthology.orchestra.run.smithi119:> true
2023-12-19T15:23:03.634 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com'
2023-12-19T15:23:03.634 DEBUG:teuthology.orchestra.run.smithi119:> uname -m
2023-12-19T15:23:03.687 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64
2023-12-19T15:23:03.688 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release
2023-12-19T15:23:03.742 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Red Hat Enterprise Linux"
2023-12-19T15:23:03.742 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="8.4 (Ootpa)"
2023-12-19T15:23:03.742 INFO:teuthology.orchestra.run.smithi119.stdout:ID="rhel"
2023-12-19T15:23:03.742 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="fedora"
2023-12-19T15:23:03.742 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="8.4"
2023-12-19T15:23:03.742 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el8"
2023-12-19T15:23:03.742 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-12-19T15:23:03.742 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31"
2023-12-19T15:23:03.742 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-12-19T15:23:03.742 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.redhat.com/"
2023-12-19T15:23:03.743 INFO:teuthology.orchestra.run.smithi119.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-12-19T15:23:03.743 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-19T15:23:03.743 INFO:teuthology.orchestra.run.smithi119.stdout:
2023-12-19T15:23:03.743 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-19T15:23:03.743 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-12-19T15:23:03.743 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-12-19T15:23:03.743 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-12-19T15:23:03.743 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server
2023-12-19T15:23:03.767 INFO:teuthology.lock.ops:Updating [smithi143.front.sepia.ceph.com]: set os type and version on server
2023-12-19T15:23:03.768 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com'
2023-12-19T15:23:03.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-19T15:23:03.836 DEBUG:teuthology.orchestra.run.smithi143:> true
2023-12-19T15:23:03.905 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com'
2023-12-19T15:23:03.905 DEBUG:teuthology.orchestra.run.smithi143:> uname -m
2023-12-19T15:23:03.958 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64
2023-12-19T15:23:03.958 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release
2023-12-19T15:23:04.010 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="Red Hat Enterprise Linux"
2023-12-19T15:23:04.011 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="8.4 (Ootpa)"
2023-12-19T15:23:04.011 INFO:teuthology.orchestra.run.smithi143.stdout:ID="rhel"
2023-12-19T15:23:04.011 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE="fedora"
2023-12-19T15:23:04.011 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="8.4"
2023-12-19T15:23:04.011 INFO:teuthology.orchestra.run.smithi143.stdout:PLATFORM_ID="platform:el8"
2023-12-19T15:23:04.011 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-12-19T15:23:04.011 INFO:teuthology.orchestra.run.smithi143.stdout:ANSI_COLOR="0;31"
2023-12-19T15:23:04.011 INFO:teuthology.orchestra.run.smithi143.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-12-19T15:23:04.011 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://www.redhat.com/"
2023-12-19T15:23:04.011 INFO:teuthology.orchestra.run.smithi143.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-12-19T15:23:04.011 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-19T15:23:04.011 INFO:teuthology.orchestra.run.smithi143.stdout:
2023-12-19T15:23:04.012 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-19T15:23:04.012 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-12-19T15:23:04.012 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-12-19T15:23:04.012 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-12-19T15:23:04.012 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server
2023-12-19T15:23:04.034 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T15:23:04.149 INFO:teuthology.dispatcher.supervisor:Running job 7496999
2023-12-19T15:23:04.150 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-12-19_12:42:58-fs-rishabh-mdscap-pacific-testing-default-smithi/7496999 --name rishabh-2023-12-19_12:42:58-fs-rishabh-mdscap-pacific-testing-default-smithi --description fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} -- /home/teuthworker/archive/rishabh-2023-12-19_12:42:58-fs-rishabh-mdscap-pacific-testing-default-smithi/7496999/orig.config.yaml
2023-12-19T15:23:04.154 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-12-19_12:42:58-fs-rishabh-mdscap-pacific-testing-default-smithi/7496999
2023-12-19T15:23:04.155 INFO:teuthology.dispatcher.supervisor:Job PID: 2952083
2023-12-19T15:23:04.155 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-12-19T15:25:04.254 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T15:27:04.422 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T15:29:04.594 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T15:31:04.774 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T15:33:04.950 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-19T15:33:05.018 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2023-12-19T15:33:05.125 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-12-19T15:33:05.125 INFO:teuthology.nuke:Checking targets against current locks
2023-12-19T15:33:05.168 DEBUG:teuthology.nuke:shortname: smithi119
2023-12-19T15:33:05.169 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-19T15:33:05.189 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-19_12:42:58-fs-rishabh-mdscap-pacific-testing-default-smithi/7496999', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 15:18:08.813996', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDHe3Q0xUaP8bgHV3A5qZR4ToWkXgqbLhoYc6/zznsryRgM9i2hSWgshBnx8JWroZDXgSqSHgwEnxEVLxtmTl58='}
2023-12-19T15:33:05.192 DEBUG:teuthology.nuke:shortname: smithi143
2023-12-19T15:33:05.193 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-19T15:33:05.212 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-19_12:42:58-fs-rishabh-mdscap-pacific-testing-default-smithi/7496999', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 15:18:08.815502', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDULA2+BfGkBqNds2bgXDbNsXc5vkg9A7+H9TeN5YUpeqolSajiS8fQoHVhSmK2ynQw3wm2noG/ol7BVg33h29U='}
2023-12-19T15:33:05.239 INFO:teuthology.orchestra.console.smithi119:Power off
2023-12-19T15:33:05.239 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-19T15:33:05.263 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off
2023-12-19T15:33:05.263 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T15:33:05.277 INFO:teuthology.orchestra.console.smithi143:Power off
2023-12-19T15:33:05.277 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-19T15:33:05.287 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off
2023-12-19T15:33:05.300 DEBUG:teuthology.orchestra.console.smithi143:power off output: Chassis Power Control: Down/Off
2023-12-19T15:33:05.301 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-19T15:33:05.325 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is off
2023-12-19T15:33:05.389 INFO:teuthology.orchestra.console.smithi119:Power off completed
2023-12-19T15:33:05.426 INFO:teuthology.orchestra.console.smithi143:Power off completed
2023-12-19T15:33:05.533 INFO:teuthology.lock.ops:unlocked: smithi119.front.sepia.ceph.com
2023-12-19T15:33:05.567 INFO:teuthology.lock.ops:unlocked: smithi143.front.sepia.ceph.com