2023-11-29T09:49:24.924 INFO:root:teuthology version: 0.0.1.dev212+g735a43c
2023-11-29T09:49:24.925 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T09:49:25.065 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f46575dad30>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f46586c9550>, signals=[15])
2023-11-29T09:49:25.066 INFO:teuthology.lock.ops:Start node 'smithi091.front.sepia.ceph.com' reimaging
2023-11-29T09:49:25.066 INFO:teuthology.lock.ops:Updating [smithi091.front.sepia.ceph.com]: reset os type and version on server
2023-11-29T09:49:25.066 INFO:teuthology.lock.ops:Updating smithi091.front.sepia.ceph.com on lock server
2023-11-29T09:49:25.082 INFO:teuthology.lock.ops:Start node 'smithi183.front.sepia.ceph.com' reimaging
2023-11-29T09:49:25.083 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: reset os type and version on server
2023-11-29T09:49:25.083 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server
2023-11-29T09:49:25.207 DEBUG:teuthology.provision.fog.smithi091:Requesting image smithi_rhel_8.6 (ID 5)
2023-11-29T09:49:25.258 INFO:teuthology.provision.fog.smithi091:Scheduling deploy of rhel 8.6
2023-11-29T09:49:25.321 DEBUG:teuthology.provision.fog.smithi183:Requesting image smithi_rhel_8.6 (ID 5)
2023-11-29T09:49:25.372 INFO:teuthology.provision.fog.smithi183:Scheduling deploy of rhel 8.6
2023-11-29T09:49:26.056 INFO:teuthology.orchestra.console.smithi091:Power off
2023-11-29T09:49:26.056 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-29T09:49:26.076 DEBUG:teuthology.orchestra.console.smithi091:power off output: Chassis Power Control: Down/Off
2023-11-29T09:49:26.076 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T09:49:26.096 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is off
2023-11-29T09:49:26.156 INFO:teuthology.orchestra.console.smithi183:Power off
2023-11-29T09:49:26.157 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-29T09:49:26.176 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off
2023-11-29T09:49:26.176 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T09:49:26.195 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off
2023-11-29T09:49:26.197 INFO:teuthology.orchestra.console.smithi091:Power off completed
2023-11-29T09:49:26.296 INFO:teuthology.orchestra.console.smithi183:Power off completed
2023-11-29T09:49:26.298 INFO:teuthology.orchestra.console.smithi091:Power on
2023-11-29T09:49:26.298 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-29T09:49:26.319 DEBUG:teuthology.orchestra.console.smithi091:power on output: Chassis Power Control: Up/On
2023-11-29T09:49:26.319 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T09:49:26.338 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is off
2023-11-29T09:49:26.397 INFO:teuthology.orchestra.console.smithi183:Power on
2023-11-29T09:49:26.397 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-29T09:49:26.417 DEBUG:teuthology.orchestra.console.smithi183:power on output: Chassis Power Control: Up/On
2023-11-29T09:49:26.417 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T09:49:26.436 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off
2023-11-29T09:49:30.343 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T09:49:30.438 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T09:49:30.453 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is on
2023-11-29T09:49:30.547 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on
2023-11-29T09:49:30.554 INFO:teuthology.orchestra.console.smithi091:Power on completed
2023-11-29T09:49:30.648 INFO:teuthology.orchestra.console.smithi183:Power on completed
2023-11-29T09:49:30.656 INFO:teuthology.provision.fog.smithi091:Waiting for deploy to finish
2023-11-29T09:49:30.749 INFO:teuthology.provision.fog.smithi183:Waiting for deploy to finish
2023-11-29T09:54:03.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:54:03.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:54:06.690 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2023-11-29T09:54:06.691 WARNING:teuthology.provision.fog.smithi091:[Errno None] Unable to connect to port 22 on 172.21.15.91
2023-11-29T09:54:12.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:54:12.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:54:15.750 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2023-11-29T09:54:15.751 WARNING:teuthology.provision.fog.smithi091:[Errno None] Unable to connect to port 22 on 172.21.15.91
2023-11-29T09:54:21.757 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:54:21.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:54:24.838 WARNING:teuthology.provision.fog.smithi091:[Errno None] Unable to connect to port 22 on 172.21.15.91
2023-11-29T09:54:24.838 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2023-11-29T09:54:30.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:54:30.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:54:33.926 WARNING:teuthology.provision.fog.smithi091:[Errno None] Unable to connect to port 22 on 172.21.15.91
2023-11-29T09:54:33.927 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2023-11-29T09:54:39.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:54:39.932 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:54:43.014 WARNING:teuthology.provision.fog.smithi091:[Errno None] Unable to connect to port 22 on 172.21.15.91
2023-11-29T09:54:43.015 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2023-11-29T09:54:49.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:54:49.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:54:52.102 WARNING:teuthology.provision.fog.smithi091:[Errno None] Unable to connect to port 22 on 172.21.15.91
2023-11-29T09:54:52.103 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2023-11-29T09:54:58.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:54:58.108 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:55:16.450 WARNING:teuthology.provision.fog.smithi091:[Errno None] Unable to connect to port 22 on 172.21.15.91
2023-11-29T09:55:22.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:55:25.510 WARNING:teuthology.provision.fog.smithi091:[Errno None] Unable to connect to port 22 on 172.21.15.91
2023-11-29T09:55:31.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:55:33.064 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-29T09:55:33.064 DEBUG:teuthology.orchestra.run.smithi091:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-29T09:55:33.337 INFO:teuthology.provision.fog.smithi091:Node is ready
2023-11-29T09:55:33.337 DEBUG:teuthology.orchestra.run.smithi091:> hostname
2023-11-29T09:55:33.353 INFO:teuthology.orchestra.run.smithi091.stdout:smithi022
2023-11-29T09:55:33.353 DEBUG:teuthology.orchestra.run.smithi091:> grep smithi022 /etc/hosts
2023-11-29T09:55:33.410 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022
2023-11-29T09:55:33.410 DEBUG:teuthology.orchestra.run.smithi091:> sudo hostname smithi091
2023-11-29T09:55:33.530 DEBUG:teuthology.orchestra.run.smithi091:> sudo sed -i -e 's/smithi022/smithi091/g' /etc/hosts
2023-11-29T09:55:33.556 DEBUG:teuthology.orchestra.run.smithi091:> sudo sed -i -e 's/172.21.15.22/172.21.15.91/g' /etc/hosts
2023-11-29T09:55:33.623 DEBUG:teuthology.orchestra.run.smithi091:> sudo sed -i -e 's/smithi022/smithi091/g' /etc/hostname
2023-11-29T09:55:33.690 DEBUG:teuthology.orchestra.run.smithi091:> sudo hostname smithi091
2023-11-29T09:55:33.756 DEBUG:teuthology.orchestra.run.smithi091:> cat /etc/os-release
2023-11-29T09:55:33.812 INFO:teuthology.orchestra.run.smithi091.stdout:NAME="Red Hat Enterprise Linux"
2023-11-29T09:55:33.812 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION="8.6 (Ootpa)"
2023-11-29T09:55:33.813 INFO:teuthology.orchestra.run.smithi091.stdout:ID="rhel"
2023-11-29T09:55:33.813 INFO:teuthology.orchestra.run.smithi091.stdout:ID_LIKE="fedora"
2023-11-29T09:55:33.813 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION_ID="8.6"
2023-11-29T09:55:33.813 INFO:teuthology.orchestra.run.smithi091.stdout:PLATFORM_ID="platform:el8"
2023-11-29T09:55:33.813 INFO:teuthology.orchestra.run.smithi091.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-11-29T09:55:33.813 INFO:teuthology.orchestra.run.smithi091.stdout:ANSI_COLOR="0;31"
2023-11-29T09:55:33.813 INFO:teuthology.orchestra.run.smithi091.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-11-29T09:55:33.813 INFO:teuthology.orchestra.run.smithi091.stdout:HOME_URL="https://www.redhat.com/"
2023-11-29T09:55:33.813 INFO:teuthology.orchestra.run.smithi091.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-29T09:55:33.813 INFO:teuthology.orchestra.run.smithi091.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-29T09:55:33.813 INFO:teuthology.orchestra.run.smithi091.stdout:
2023-11-29T09:55:33.813 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-29T09:55:33.813 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-11-29T09:55:33.813 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-29T09:55:33.814 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-11-29T09:55:33.814 INFO:teuthology.provision.fog.smithi091:Deploy complete!
2023-11-29T09:55:33.816 DEBUG:teuthology.parallel:result is None
2023-11-29T09:55:58.134 WARNING:teuthology.provision.fog.smithi183:timed out
2023-11-29T09:56:04.139 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:56:04.496 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-29T09:56:04.496 DEBUG:teuthology.orchestra.run.smithi183:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-29T09:56:04.759 INFO:teuthology.provision.fog.smithi183:Node is ready
2023-11-29T09:56:04.760 DEBUG:teuthology.orchestra.run.smithi183:> hostname
2023-11-29T09:56:04.777 INFO:teuthology.orchestra.run.smithi183.stdout:smithi183
2023-11-29T09:56:04.777 DEBUG:teuthology.orchestra.run.smithi183:> grep smithi183 /etc/hosts
2023-11-29T09:56:04.837 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.15.183 smithi183.front.sepia.ceph.com smithi183
2023-11-29T09:56:04.837 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname smithi183
2023-11-29T09:56:04.948 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/smithi183/smithi183/g' /etc/hosts
2023-11-29T09:56:04.976 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/172.21.15.183/172.21.15.183/g' /etc/hosts
2023-11-29T09:56:05.044 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/smithi183/smithi183/g' /etc/hostname
2023-11-29T09:56:05.111 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname smithi183
2023-11-29T09:56:05.178 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release
2023-11-29T09:56:05.237 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="Red Hat Enterprise Linux"
2023-11-29T09:56:05.237 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="8.6 (Ootpa)"
2023-11-29T09:56:05.238 INFO:teuthology.orchestra.run.smithi183.stdout:ID="rhel"
2023-11-29T09:56:05.238 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="fedora"
2023-11-29T09:56:05.238 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="8.6"
2023-11-29T09:56:05.238 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el8"
2023-11-29T09:56:05.238 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-11-29T09:56:05.238 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31"
2023-11-29T09:56:05.238 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-11-29T09:56:05.238 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://www.redhat.com/"
2023-11-29T09:56:05.238 INFO:teuthology.orchestra.run.smithi183.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-29T09:56:05.238 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-29T09:56:05.238 INFO:teuthology.orchestra.run.smithi183.stdout:
2023-11-29T09:56:05.239 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-29T09:56:05.239 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-11-29T09:56:05.239 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-29T09:56:05.239 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-11-29T09:56:05.239 INFO:teuthology.provision.fog.smithi183:Deploy complete!
2023-11-29T09:56:05.240 DEBUG:teuthology.parallel:result is None
2023-11-29T09:56:05.454 INFO:teuthology.lock.ops:Checking smithi091.front.sepia.ceph.com
2023-11-29T09:56:05.455 INFO:teuthology.lock.ops:New key found. Updating...
2023-11-29T09:56:05.478 INFO:teuthology.lock.ops:Checking smithi183.front.sepia.ceph.com
2023-11-29T09:56:05.478 INFO:teuthology.lock.ops:New key found. Updating...
2023-11-29T09:56:05.496 INFO:teuthology.lock.ops:Updating [smithi091.front.sepia.ceph.com]: set os type and version on server
2023-11-29T09:56:05.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi091.front.sepia.ceph.com'
2023-11-29T09:56:05.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi091.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:56:05.569 DEBUG:teuthology.orchestra.run.smithi091:> true
2023-11-29T09:56:05.700 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi091.front.sepia.ceph.com'
2023-11-29T09:56:05.700 DEBUG:teuthology.orchestra.run.smithi091:> uname -m
2023-11-29T09:56:05.758 INFO:teuthology.orchestra.run.smithi091.stdout:x86_64
2023-11-29T09:56:05.758 DEBUG:teuthology.orchestra.run.smithi091:> cat /etc/os-release
2023-11-29T09:56:05.818 INFO:teuthology.orchestra.run.smithi091.stdout:NAME="Red Hat Enterprise Linux"
2023-11-29T09:56:05.818 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION="8.6 (Ootpa)"
2023-11-29T09:56:05.818 INFO:teuthology.orchestra.run.smithi091.stdout:ID="rhel"
2023-11-29T09:56:05.818 INFO:teuthology.orchestra.run.smithi091.stdout:ID_LIKE="fedora"
2023-11-29T09:56:05.818 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION_ID="8.6"
2023-11-29T09:56:05.818 INFO:teuthology.orchestra.run.smithi091.stdout:PLATFORM_ID="platform:el8"
2023-11-29T09:56:05.818 INFO:teuthology.orchestra.run.smithi091.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-11-29T09:56:05.818 INFO:teuthology.orchestra.run.smithi091.stdout:ANSI_COLOR="0;31"
2023-11-29T09:56:05.819 INFO:teuthology.orchestra.run.smithi091.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-11-29T09:56:05.819 INFO:teuthology.orchestra.run.smithi091.stdout:HOME_URL="https://www.redhat.com/"
2023-11-29T09:56:05.819 INFO:teuthology.orchestra.run.smithi091.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-29T09:56:05.819 INFO:teuthology.orchestra.run.smithi091.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-29T09:56:05.819 INFO:teuthology.orchestra.run.smithi091.stdout:
2023-11-29T09:56:05.819 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-29T09:56:05.819 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-11-29T09:56:05.819 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-29T09:56:05.819 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-11-29T09:56:05.820 INFO:teuthology.lock.ops:Updating smithi091.front.sepia.ceph.com on lock server
2023-11-29T09:56:05.844 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: set os type and version on server
2023-11-29T09:56:05.845 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com'
2023-11-29T09:56:05.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T09:56:05.911 DEBUG:teuthology.orchestra.run.smithi183:> true
2023-11-29T09:56:05.978 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com'
2023-11-29T09:56:05.978 DEBUG:teuthology.orchestra.run.smithi183:> uname -m
2023-11-29T09:56:06.036 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64
2023-11-29T09:56:06.037 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release
2023-11-29T09:56:06.095 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="Red Hat Enterprise Linux"
2023-11-29T09:56:06.095 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="8.6 (Ootpa)"
2023-11-29T09:56:06.095 INFO:teuthology.orchestra.run.smithi183.stdout:ID="rhel"
2023-11-29T09:56:06.095 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="fedora"
2023-11-29T09:56:06.095 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="8.6"
2023-11-29T09:56:06.096 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el8"
2023-11-29T09:56:06.096 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-11-29T09:56:06.096 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31"
2023-11-29T09:56:06.096 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-11-29T09:56:06.096 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://www.redhat.com/"
2023-11-29T09:56:06.096 INFO:teuthology.orchestra.run.smithi183.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-11-29T09:56:06.096 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-29T09:56:06.096 INFO:teuthology.orchestra.run.smithi183.stdout:
2023-11-29T09:56:06.096 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-11-29T09:56:06.096 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-11-29T09:56:06.096 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-11-29T09:56:06.096 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-11-29T09:56:06.097 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server
2023-11-29T09:56:06.164 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T09:56:06.320 INFO:teuthology.dispatcher.supervisor:Running job 7470597
2023-11-29T09:56:06.320 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/virtualenv/bin/teuthology -v --owner mchangir --archive /home/teuthworker/archive/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7470597 --name mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} -- /home/teuthworker/archive/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7470597/orig.config.yaml
2023-11-29T09:56:06.335 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7470597
2023-11-29T09:56:06.335 INFO:teuthology.dispatcher.supervisor:Job PID: 2241723
2023-11-29T09:56:06.335 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-11-29T09:58:06.435 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:00:06.632 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:02:06.813 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:04:06.983 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:06:07.173 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:08:07.339 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:10:07.519 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:12:07.688 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:14:07.863 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:16:08.029 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:18:08.195 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:20:08.364 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:22:08.527 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:24:08.668 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:26:08.806 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:28:08.973 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:30:09.126 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:32:09.278 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:34:09.444 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:36:09.611 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:38:09.785 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:40:09.943 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:42:10.103 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:44:10.275 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:46:10.439 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:48:10.603 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:50:10.763 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:52:10.924 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:54:11.079 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:56:11.223 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T10:58:11.388 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:00:11.557 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:02:11.731 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:04:11.903 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:06:12.075 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:08:12.247 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:10:12.398 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:12:12.567 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:14:12.730 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:16:12.888 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:18:12.988 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:20:13.108 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:22:13.287 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:24:13.435 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:26:13.605 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:28:13.749 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:30:13.915 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:32:14.054 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:34:14.157 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:36:14.275 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:38:14.376 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:40:14.537 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:42:14.659 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:44:14.830 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:46:14.990 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:48:15.142 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:50:15.319 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:52:15.491 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:54:15.663 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:56:15.831 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T11:56:15.897 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2023-11-29T11:56:16.061 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-11-29T11:56:16.062 INFO:teuthology.nuke:Checking targets against current locks
2023-11-29T11:56:16.112 DEBUG:teuthology.nuke:shortname: smithi091
2023-11-29T11:56:16.112 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-29T11:56:16.136 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi091.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7470597', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-29 09:49:24.163469', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}
2023-11-29T11:56:16.140 DEBUG:teuthology.nuke:shortname: smithi183
2023-11-29T11:56:16.141 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-29T11:56:16.167 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7470597', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-29 09:49:24.161821', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}
2023-11-29T11:56:16.192 INFO:teuthology.orchestra.console.smithi091:Power off
2023-11-29T11:56:16.192 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-29T11:56:16.217 DEBUG:teuthology.orchestra.console.smithi091:power off output: Chassis Power Control: Down/Off
2023-11-29T11:56:16.217 DEBUG:teuthology.orchestra.console.smithi091:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T11:56:16.232 INFO:teuthology.orchestra.console.smithi183:Power off
2023-11-29T11:56:16.233 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-29T11:56:16.241 DEBUG:teuthology.orchestra.console.smithi091:check power output: Chassis Power is off
2023-11-29T11:56:16.255 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off
2023-11-29T11:56:16.255 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T11:56:16.278 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off
2023-11-29T11:56:16.342 INFO:teuthology.orchestra.console.smithi091:Power off completed
2023-11-29T11:56:16.379 INFO:teuthology.orchestra.console.smithi183:Power off completed
2023-11-29T11:56:16.488 INFO:teuthology.lock.ops:unlocked: smithi091.front.sepia.ceph.com
2023-11-29T11:56:16.522 INFO:teuthology.lock.ops:unlocked: smithi183.front.sepia.ceph.com