2024-01-27T00:45:51.409 INFO:root:teuthology version: 0.0.1.dev251+gc856e0f
2024-01-27T00:45:51.409 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T00:45:51.547 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f18bb482f70>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f18bc4e8c10>, signals=[15])
2024-01-27T00:45:51.548 INFO:teuthology.lock.ops:Start node 'smithi161.front.sepia.ceph.com' reimaging
2024-01-27T00:45:51.548 INFO:teuthology.lock.ops:Updating [smithi161.front.sepia.ceph.com]: reset os type and version on server
2024-01-27T00:45:51.548 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server
2024-01-27T00:45:51.563 INFO:teuthology.lock.ops:Start node 'smithi178.front.sepia.ceph.com' reimaging
2024-01-27T00:45:51.563 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: reset os type and version on server
2024-01-27T00:45:51.563 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server
2024-01-27T00:45:51.693 DEBUG:teuthology.provision.fog.smithi161:Requesting image smithi_rhel_8.6 (ID 5)
2024-01-27T00:45:51.699 DEBUG:teuthology.provision.fog.smithi178:Requesting image smithi_rhel_8.6 (ID 5)
2024-01-27T00:45:51.740 INFO:teuthology.provision.fog.smithi161:Scheduling deploy of rhel 8.6
2024-01-27T00:45:51.745 INFO:teuthology.provision.fog.smithi178:Scheduling deploy of rhel 8.6
2024-01-27T00:45:52.000 INFO:teuthology.orchestra.console.smithi178:Power off
2024-01-27T00:45:52.001 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-27T00:45:52.015 INFO:teuthology.orchestra.console.smithi161:Power off
2024-01-27T00:45:52.015 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-27T00:45:52.023 DEBUG:teuthology.orchestra.console.smithi178:power off output: Chassis Power Control: Down/Off
2024-01-27T00:45:52.024 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T00:45:52.037 DEBUG:teuthology.orchestra.console.smithi161:power off output: Chassis Power Control: Down/Off
2024-01-27T00:45:52.037 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T00:45:52.047 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on
2024-01-27T00:45:52.057 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on
2024-01-27T00:45:56.052 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T00:45:56.059 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T00:45:56.162 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on
2024-01-27T00:45:56.168 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on
2024-01-27T00:46:00.164 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T00:46:00.171 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T00:46:00.273 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off
2024-01-27T00:46:00.280 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on
2024-01-27T00:46:00.375 INFO:teuthology.orchestra.console.smithi178:Power off completed
2024-01-27T00:46:00.476 INFO:teuthology.orchestra.console.smithi178:Power on
2024-01-27T00:46:00.476 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-27T00:46:00.497 DEBUG:teuthology.orchestra.console.smithi178:power on output: Chassis Power Control: Up/On
2024-01-27T00:46:00.498 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T00:46:00.522 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off
2024-01-27T00:46:04.285 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T00:46:04.394 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off
2024-01-27T00:46:04.495 INFO:teuthology.orchestra.console.smithi161:Power off completed
2024-01-27T00:46:04.523 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T00:46:04.596 INFO:teuthology.orchestra.console.smithi161:Power on
2024-01-27T00:46:04.596 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-27T00:46:04.620 DEBUG:teuthology.orchestra.console.smithi161:power on output: Chassis Power Control: Up/On
2024-01-27T00:46:04.620 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T00:46:04.632 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off
2024-01-27T00:46:04.646 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off
2024-01-27T00:46:08.637 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T00:46:08.646 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T00:46:08.755 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on
2024-01-27T00:46:08.787 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on
2024-01-27T00:46:08.856 INFO:teuthology.orchestra.console.smithi161:Power on completed
2024-01-27T00:46:08.888 INFO:teuthology.orchestra.console.smithi178:Power on completed
2024-01-27T00:46:08.957 INFO:teuthology.provision.fog.smithi161:Waiting for deploy to finish
2024-01-27T00:46:08.988 INFO:teuthology.provision.fog.smithi178:Waiting for deploy to finish
2024-01-27T00:48:54.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:48:58.051 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-27T00:49:04.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:49:07.136 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-27T00:49:10.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:49:13.084 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178
2024-01-27T00:49:13.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:49:16.192 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-27T00:49:19.088 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:49:22.140 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178
2024-01-27T00:49:22.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:49:25.244 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-27T00:49:28.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:49:31.196 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178
2024-01-27T00:49:31.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:49:34.300 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-27T00:49:37.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:49:40.253 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178
2024-01-27T00:49:40.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:49:43.356 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-27T00:49:46.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:49:49.312 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178
2024-01-27T00:49:49.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:49:52.416 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2024-01-27T00:49:55.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:49:58.396 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178
2024-01-27T00:49:58.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:50:04.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:50:07.452 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178
2024-01-27T00:50:13.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:50:31.868 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178
2024-01-27T00:50:37.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:50:40.928 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178
2024-01-27T00:50:46.932 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:50:50.012 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178
2024-01-27T00:50:56.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:50:56.749 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-27T00:50:56.750 DEBUG:teuthology.orchestra.run.smithi178:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-27T00:50:58.418 WARNING:teuthology.provision.fog.smithi161:timed out
2024-01-27T00:51:04.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:51:04.625 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-27T00:51:04.626 DEBUG:teuthology.orchestra.run.smithi161:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-27T00:51:04.867 INFO:teuthology.provision.fog.smithi161:Node is ready
2024-01-27T00:51:04.867 DEBUG:teuthology.orchestra.run.smithi161:> hostname
2024-01-27T00:51:04.884 INFO:teuthology.orchestra.run.smithi161.stdout:smithi161
2024-01-27T00:51:04.885 DEBUG:teuthology.orchestra.run.smithi161:> grep smithi161 /etc/hosts
2024-01-27T00:51:04.942 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161 smithi161.front.sepia.ceph.com smithi161
2024-01-27T00:51:04.943 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname smithi161
2024-01-27T00:51:05.054 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/smithi161/smithi161/g' /etc/hosts
2024-01-27T00:51:05.081 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/172.21.15.161/172.21.15.161/g' /etc/hosts
2024-01-27T00:51:05.149 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/smithi161/smithi161/g' /etc/hostname
2024-01-27T00:51:05.216 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname smithi161
2024-01-27T00:51:05.282 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release
2024-01-27T00:51:05.338 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Red Hat Enterprise Linux"
2024-01-27T00:51:05.338 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8.6 (Ootpa)"
2024-01-27T00:51:05.339 INFO:teuthology.orchestra.run.smithi161.stdout:ID="rhel"
2024-01-27T00:51:05.339 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="fedora"
2024-01-27T00:51:05.339 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8.6"
2024-01-27T00:51:05.339 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8"
2024-01-27T00:51:05.339 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2024-01-27T00:51:05.339 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31"
2024-01-27T00:51:05.339 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2024-01-27T00:51:05.339 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.redhat.com/"
2024-01-27T00:51:05.339 INFO:teuthology.orchestra.run.smithi161.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2024-01-27T00:51:05.339 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-27T00:51:05.339 INFO:teuthology.orchestra.run.smithi161.stdout:
2024-01-27T00:51:05.339 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2024-01-27T00:51:05.339 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2024-01-27T00:51:05.339 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2024-01-27T00:51:05.340 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2024-01-27T00:51:05.340 INFO:teuthology.provision.fog.smithi161:Deploy complete!
2024-01-27T00:51:05.342 DEBUG:teuthology.parallel:result is None
2024-01-27T00:51:12.216 INFO:teuthology.provision.fog.smithi178:Node is ready
2024-01-27T00:51:12.216 DEBUG:teuthology.orchestra.run.smithi178:> hostname
2024-01-27T00:51:12.237 INFO:teuthology.orchestra.run.smithi178.stdout:smithi178.front.sepia.ceph.com
2024-01-27T00:51:12.238 DEBUG:teuthology.orchestra.run.smithi178:> grep smithi178.front.sepia.ceph.com /etc/hosts
2024-01-27T00:51:12.297 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178 smithi178.front.sepia.ceph.com smithi178
2024-01-27T00:51:12.297 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname smithi178
2024-01-27T00:51:12.433 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i -e 's/smithi178.front.sepia.ceph.com/smithi178/g' /etc/hosts
2024-01-27T00:51:12.463 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i -e 's/172.21.15.178/172.21.15.178/g' /etc/hosts
2024-01-27T00:51:12.532 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i -e 's/smithi178.front.sepia.ceph.com/smithi178/g' /etc/hostname
2024-01-27T00:51:12.600 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname smithi178
2024-01-27T00:51:12.667 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release
2024-01-27T00:51:12.724 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Red Hat Enterprise Linux"
2024-01-27T00:51:12.724 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="8.6 (Ootpa)"
2024-01-27T00:51:12.725 INFO:teuthology.orchestra.run.smithi178.stdout:ID="rhel"
2024-01-27T00:51:12.725 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="fedora"
2024-01-27T00:51:12.725 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="8.6"
2024-01-27T00:51:12.725 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el8"
2024-01-27T00:51:12.725 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2024-01-27T00:51:12.725 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31"
2024-01-27T00:51:12.725 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2024-01-27T00:51:12.725 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.redhat.com/"
2024-01-27T00:51:12.725 INFO:teuthology.orchestra.run.smithi178.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2024-01-27T00:51:12.725 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-27T00:51:12.725 INFO:teuthology.orchestra.run.smithi178.stdout:
2024-01-27T00:51:12.725 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2024-01-27T00:51:12.725 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2024-01-27T00:51:12.726 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2024-01-27T00:51:12.726 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2024-01-27T00:51:12.726 INFO:teuthology.provision.fog.smithi178:Deploy complete!
2024-01-27T00:51:12.726 DEBUG:teuthology.parallel:result is None
2024-01-27T00:51:12.873 INFO:teuthology.lock.ops:Checking smithi161.front.sepia.ceph.com
2024-01-27T00:51:12.873 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-27T00:51:12.899 INFO:teuthology.lock.ops:Checking smithi178.front.sepia.ceph.com
2024-01-27T00:51:12.899 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-27T00:51:12.925 INFO:teuthology.lock.ops:Updating [smithi161.front.sepia.ceph.com]: set os type and version on server
2024-01-27T00:51:12.925 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com'
2024-01-27T00:51:12.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:51:13.002 DEBUG:teuthology.orchestra.run.smithi161:> true
2024-01-27T00:51:13.031 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com'
2024-01-27T00:51:13.031 DEBUG:teuthology.orchestra.run.smithi161:> uname -m
2024-01-27T00:51:13.088 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64
2024-01-27T00:51:13.088 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release
2024-01-27T00:51:13.144 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Red Hat Enterprise Linux"
2024-01-27T00:51:13.145 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8.6 (Ootpa)"
2024-01-27T00:51:13.145 INFO:teuthology.orchestra.run.smithi161.stdout:ID="rhel"
2024-01-27T00:51:13.145 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="fedora"
2024-01-27T00:51:13.145 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8.6"
2024-01-27T00:51:13.145 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8"
2024-01-27T00:51:13.145 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2024-01-27T00:51:13.145 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31"
2024-01-27T00:51:13.145 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2024-01-27T00:51:13.145 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.redhat.com/"
2024-01-27T00:51:13.145 INFO:teuthology.orchestra.run.smithi161.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2024-01-27T00:51:13.145 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-27T00:51:13.146 INFO:teuthology.orchestra.run.smithi161.stdout:
2024-01-27T00:51:13.146 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2024-01-27T00:51:13.146 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2024-01-27T00:51:13.146 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2024-01-27T00:51:13.146 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2024-01-27T00:51:13.146 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server
2024-01-27T00:51:13.166 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: set os type and version on server
2024-01-27T00:51:13.166 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com'
2024-01-27T00:51:13.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-27T00:51:13.237 DEBUG:teuthology.orchestra.run.smithi178:> true
2024-01-27T00:51:13.261 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com'
2024-01-27T00:51:13.261 DEBUG:teuthology.orchestra.run.smithi178:> uname -m
2024-01-27T00:51:13.317 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64
2024-01-27T00:51:13.318 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release
2024-01-27T00:51:13.373 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Red Hat Enterprise Linux"
2024-01-27T00:51:13.374 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="8.6 (Ootpa)"
2024-01-27T00:51:13.374 INFO:teuthology.orchestra.run.smithi178.stdout:ID="rhel"
2024-01-27T00:51:13.374 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="fedora"
2024-01-27T00:51:13.374 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="8.6"
2024-01-27T00:51:13.374 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el8"
2024-01-27T00:51:13.374 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2024-01-27T00:51:13.374 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31"
2024-01-27T00:51:13.374 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2024-01-27T00:51:13.374 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.redhat.com/"
2024-01-27T00:51:13.374 INFO:teuthology.orchestra.run.smithi178.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2024-01-27T00:51:13.374 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-27T00:51:13.374 INFO:teuthology.orchestra.run.smithi178.stdout:
2024-01-27T00:51:13.374 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2024-01-27T00:51:13.375 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2024-01-27T00:51:13.375 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2024-01-27T00:51:13.375 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2024-01-27T00:51:13.375 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server
2024-01-27T00:51:13.400 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T00:51:13.497 INFO:teuthology.dispatcher.supervisor:Running job 7533476
2024-01-27T00:51:13.497 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533476 --name yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi --description fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} -- /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533476/orig.config.yaml
2024-01-27T00:51:13.500 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533476
2024-01-27T00:51:13.501 INFO:teuthology.dispatcher.supervisor:Job PID: 733832
2024-01-27T00:51:13.501 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-01-27T00:53:13.600 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T00:55:13.760 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T00:57:13.925 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T00:59:14.092 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:01:14.254 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:03:14.399 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:05:14.564 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:07:14.732 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:09:14.900 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:11:15.037 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:13:15.284 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:15:15.458 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:17:15.626 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:19:15.778 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:21:15.944 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:23:16.116 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:25:16.292 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:27:16.465 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:29:16.631 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:31:16.796 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:33:16.964 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:35:17.132 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-27T01:35:17.207 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2024-01-27T01:35:17.382 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2024-01-27T01:35:17.383 INFO:teuthology.nuke:Checking targets against current locks
2024-01-27T01:35:17.435 DEBUG:teuthology.nuke:shortname: smithi161
2024-01-27T01:35:17.435 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-27T01:35:17.457 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533476', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-27 00:45:50.283603', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA2JNME6vp7z/Vz10mlgTB5uciWK2sXAVdht8EuvuuWALwaOFWUmeTPFg5edpcBBcxlv67rOw/GiLXap5s0dJos='}
2024-01-27T01:35:17.460 DEBUG:teuthology.nuke:shortname: smithi178
2024-01-27T01:35:17.461 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-27T01:35:17.484 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-01-26_01:07:29-fs-wip-yuri4-testing-2024-01-25-1331-quincy-distro-default-smithi/7533476', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-27 00:45:50.281967', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMP4+ItDiy3zpHyX1zAfzbQrnn1hd4C/cQqbYjL5vaQj9dK+SJlqrtZlLLRqPW9+P6DjI/mQ6X4suWmF+lKlrYA='}
2024-01-27T01:35:17.515 INFO:teuthology.orchestra.console.smithi161:Power off
2024-01-27T01:35:17.515 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-27T01:35:17.530 INFO:teuthology.orchestra.console.smithi178:Power off
2024-01-27T01:35:17.531 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-27T01:35:17.539 DEBUG:teuthology.orchestra.console.smithi161:power off output: Chassis Power Control: Down/Off
2024-01-27T01:35:17.540 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T01:35:17.552 DEBUG:teuthology.orchestra.console.smithi178:power off output: Chassis Power Control: Down/Off
2024-01-27T01:35:17.552 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-27T01:35:17.563 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off
2024-01-27T01:35:17.573 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off
2024-01-27T01:35:17.664 INFO:teuthology.orchestra.console.smithi161:Power off completed
2024-01-27T01:35:17.674 INFO:teuthology.orchestra.console.smithi178:Power off completed
2024-01-27T01:35:17.812 INFO:teuthology.lock.ops:unlocked: smithi161.front.sepia.ceph.com
2024-01-27T01:35:17.820 INFO:teuthology.lock.ops:unlocked: smithi178.front.sepia.ceph.com