2023-07-21T17:09:02.629 INFO:root:teuthology version: 0.0.1.dev153+g79491cb 2023-07-21T17:09:02.631 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T17:09:02.632 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T17:09:02.788 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe67bed8ee0>, signals=[15]) 2023-07-21T17:09:02.789 INFO:teuthology.lock.ops:Start node 'smithi119.front.sepia.ceph.com' reimaging 2023-07-21T17:09:02.790 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: reset os type and version on server 2023-07-21T17:09:02.790 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2023-07-21T17:09:12.835 INFO:teuthology.lock.ops:Start node 'smithi139.front.sepia.ceph.com' reimaging 2023-07-21T17:09:12.836 INFO:teuthology.lock.ops:Updating [smithi139.front.sepia.ceph.com]: reset os type and version on server 2023-07-21T17:09:12.836 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2023-07-21T17:09:12.837 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /host {"name": "smithi119"} 2023-07-21T17:09:22.875 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /host {"name": "smithi139"} 2023-07-21T17:09:32.913 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-21T17:09:37.938 DEBUG:teuthology.provision.fog.smithi119:Requesting image smithi_centos_9.stream (ID 10) 2023-07-21T17:09:37.938 DEBUG:teuthology.provision.fog.smithi119:FOG request: PUT /host/224 {"imageID": 10} 2023-07-21T17:09:42.971 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-21T17:09:43.749 INFO:teuthology.provision.fog.smithi119:Scheduling deploy of centos 9.stream 2023-07-21T17:09:43.749 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-07-21T17:09:43.801 DEBUG:teuthology.provision.fog.smithi139:Requesting image smithi_centos_9.stream (ID 10) 2023-07-21T17:09:43.801 DEBUG:teuthology.provision.fog.smithi139:FOG request: PUT /host/244 {"imageID": 10} 2023-07-21T17:09:43.840 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /tasktype {"name": "deploy"} 2023-07-21T17:09:43.848 INFO:teuthology.provision.fog.smithi139:Scheduling deploy of centos 9.stream 2023-07-21T17:09:43.848 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-21T17:09:43.887 DEBUG:teuthology.provision.fog.smithi119:FOG request: POST /host/224/task {"taskTypeID": 1} 2023-07-21T17:09:43.917 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /tasktype {"name": "deploy"} 2023-07-21T17:09:43.951 DEBUG:teuthology.provision.fog.smithi139:FOG request: POST /host/244/task {"taskTypeID": 1} 2023-07-21T17:09:43.985 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-07-21T17:09:44.000 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-21T17:09:47.672 INFO:teuthology.orchestra.console:Power off smithi139 2023-07-21T17:09:47.673 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T17:09:47.687 INFO:teuthology.orchestra.console:Power off smithi119 2023-07-21T17:09:47.688 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T17:09:47.696 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:09:47.716 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:09:51.728 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:09:51.742 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:09:56.845 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:09:56.853 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:01.956 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:01.966 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:02.168 INFO:teuthology.orchestra.console:Power off for smithi139 completed 2023-07-21T17:10:02.269 INFO:teuthology.orchestra.console:Power on smithi139 2023-07-21T17:10:02.270 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-21T17:10:07.079 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:12.190 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:12.400 INFO:teuthology.orchestra.console:Power off for smithi119 completed 2023-07-21T17:10:12.501 INFO:teuthology.orchestra.console:Power on smithi119 2023-07-21T17:10:12.502 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-21T17:10:17.302 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:17.526 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:22.316 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:22.537 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:27.427 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:27.650 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:32.544 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:32.761 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:37.660 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:37.872 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:42.772 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:42.983 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:47.884 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:48.092 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:10:48.101 INFO:teuthology.orchestra.console:Power on for smithi139 completed 2023-07-21T17:10:48.203 INFO:teuthology.provision.fog.smithi139:Waiting for deploy to finish 2023-07-21T17:10:48.204 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-21T17:10:57.212 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:11:02.325 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:11:07.440 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:11:08.297 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-21T17:11:12.552 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:11:17.663 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:11:17.873 INFO:teuthology.orchestra.console:Power on for smithi119 completed 2023-07-21T17:11:17.974 INFO:teuthology.provision.fog.smithi119:Waiting for deploy to finish 2023-07-21T17:11:17.974 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-07-21T17:11:23.402 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-21T17:11:38.072 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-07-21T17:11:44.251 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-21T17:11:58.164 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-07-21T17:12:04.330 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-21T17:12:14.572 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-07-21T17:12:24.408 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-21T17:12:34.572 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-07-21T17:12:39.476 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-21T17:12:49.648 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None 2023-07-21T17:12:54.653 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-21T17:12:59.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T17:12:59.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T17:13:02.800 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-07-21T17:13:02.801 WARNING:teuthology.provision.fog.smithi139:[Errno None] Unable to connect to port 22 on 172.21.15.139 2023-07-21T17:13:08.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T17:13:08.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T17:13:11.856 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-07-21T17:13:11.888 WARNING:teuthology.provision.fog.smithi139:[Errno None] Unable to connect to port 22 on 172.21.15.139 2023-07-21T17:13:17.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T17:13:22.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T17:13:27.874 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-07-21T17:13:30.928 WARNING:teuthology.provision.fog.smithi139:[Errno None] Unable to connect to port 22 on 172.21.15.139 2023-07-21T17:13:33.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T17:13:38.885 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T17:13:43.887 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-07-21T17:13:46.960 WARNING:teuthology.provision.fog.smithi139:[Errno None] Unable to connect to port 22 on 172.21.15.139 2023-07-21T17:13:49.892 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T17:13:54.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T17:13:59.903 WARNING:teuthology.provision.fog.smithi119:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-07-21T17:14:05.908 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T17:14:18.352 WARNING:teuthology.provision.fog.smithi139:[Errno None] Unable to connect to port 22 on 172.21.15.139 2023-07-21T17:14:24.356 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T17:14:27.408 WARNING:teuthology.provision.fog.smithi139:[Errno None] Unable to connect to port 22 on 172.21.15.139 2023-07-21T17:14:33.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T17:14:38.645 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-21T17:14:38.646 DEBUG:teuthology.orchestra.run.smithi139:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-21T17:14:38.685 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-21T17:14:38.685 DEBUG:teuthology.orchestra.run.smithi119:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-21T17:14:53.954 INFO:teuthology.provision.fog.smithi139:Node is ready 2023-07-21T17:14:53.954 DEBUG:teuthology.orchestra.run.smithi139:> hostname 2023-07-21T17:14:53.973 INFO:teuthology.orchestra.run.smithi139.stdout:smithi139.front.sepia.ceph.com 2023-07-21T17:14:53.973 DEBUG:teuthology.orchestra.run.smithi139:> grep smithi139.front.sepia.ceph.com /etc/hosts 2023-07-21T17:14:54.031 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139 smithi139.front.sepia.ceph.com smithi139 2023-07-21T17:14:54.032 DEBUG:teuthology.orchestra.run.smithi139:> sudo hostname smithi139 2023-07-21T17:14:54.150 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i -e 's/smithi139.front.sepia.ceph.com/smithi139/g' /etc/hosts 2023-07-21T17:14:54.180 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i -e 's/172.21.15.139/172.21.15.139/g' /etc/hosts 2023-07-21T17:14:54.250 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i -e 's/smithi139.front.sepia.ceph.com/smithi139/g' /etc/hostname 2023-07-21T17:14:54.319 DEBUG:teuthology.orchestra.run.smithi139:> sudo hostname smithi139 2023-07-21T17:14:54.387 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2023-07-21T17:14:54.442 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="CentOS Stream" 2023-07-21T17:14:54.442 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="9" 2023-07-21T17:14:54.442 INFO:teuthology.orchestra.run.smithi139.stdout:ID="centos" 2023-07-21T17:14:54.442 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="rhel fedora" 2023-07-21T17:14:54.443 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="9" 2023-07-21T17:14:54.443 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el9" 2023-07-21T17:14:54.443 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-21T17:14:54.443 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2023-07-21T17:14:54.443 INFO:teuthology.orchestra.run.smithi139.stdout:LOGO="fedora-logo-icon" 2023-07-21T17:14:54.443 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-21T17:14:54.443 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://centos.org/" 2023-07-21T17:14:54.444 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T17:14:54.444 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-21T17:14:54.444 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-21T17:14:54.445 INFO:teuthology.provision.fog.smithi139:Deploy complete! 2023-07-21T17:14:54.446 DEBUG:teuthology.parallel:result is None 2023-07-21T17:14:58.652 INFO:teuthology.provision.fog.smithi119:Node is ready 2023-07-21T17:14:58.652 DEBUG:teuthology.orchestra.run.smithi119:> hostname 2023-07-21T17:14:58.670 INFO:teuthology.orchestra.run.smithi119.stdout:smithi119.front.sepia.ceph.com 2023-07-21T17:14:58.671 DEBUG:teuthology.orchestra.run.smithi119:> grep smithi119.front.sepia.ceph.com /etc/hosts 2023-07-21T17:14:58.727 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119 smithi119.front.sepia.ceph.com smithi119 2023-07-21T17:14:58.728 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname smithi119 2023-07-21T17:14:58.847 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i -e 's/smithi119.front.sepia.ceph.com/smithi119/g' /etc/hosts 2023-07-21T17:14:58.877 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i -e 's/172.21.15.119/172.21.15.119/g' /etc/hosts 2023-07-21T17:14:58.946 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i -e 's/smithi119.front.sepia.ceph.com/smithi119/g' /etc/hostname 2023-07-21T17:14:59.015 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname smithi119 2023-07-21T17:14:59.082 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2023-07-21T17:14:59.136 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2023-07-21T17:14:59.137 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="9" 2023-07-21T17:14:59.137 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2023-07-21T17:14:59.137 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2023-07-21T17:14:59.137 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="9" 2023-07-21T17:14:59.137 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el9" 2023-07-21T17:14:59.137 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-21T17:14:59.138 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2023-07-21T17:14:59.138 INFO:teuthology.orchestra.run.smithi119.stdout:LOGO="fedora-logo-icon" 2023-07-21T17:14:59.138 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-21T17:14:59.138 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2023-07-21T17:14:59.138 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T17:14:59.138 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-21T17:14:59.138 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-21T17:14:59.139 INFO:teuthology.provision.fog.smithi119:Deploy complete! 2023-07-21T17:14:59.139 DEBUG:teuthology.parallel:result is None 2023-07-21T17:15:29.309 INFO:teuthology.lock.ops:Checking smithi119.front.sepia.ceph.com 2023-07-21T17:15:29.310 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-21T17:15:34.343 INFO:teuthology.lock.ops:Checking smithi139.front.sepia.ceph.com 2023-07-21T17:15:34.344 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-21T17:15:39.444 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: set os type and version on server 2023-07-21T17:15:39.445 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-07-21T17:15:39.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T17:15:44.538 DEBUG:teuthology.orchestra.run.smithi119:> true 2023-07-21T17:15:44.714 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-07-21T17:15:44.715 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2023-07-21T17:15:44.768 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2023-07-21T17:15:44.769 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2023-07-21T17:15:44.822 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2023-07-21T17:15:44.823 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="9" 2023-07-21T17:15:44.823 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2023-07-21T17:15:44.823 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2023-07-21T17:15:44.823 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="9" 2023-07-21T17:15:44.823 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el9" 2023-07-21T17:15:44.823 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-21T17:15:44.824 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2023-07-21T17:15:44.824 INFO:teuthology.orchestra.run.smithi119.stdout:LOGO="fedora-logo-icon" 2023-07-21T17:15:44.824 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-21T17:15:44.824 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2023-07-21T17:15:44.824 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T17:15:44.824 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-21T17:15:44.824 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-21T17:15:44.825 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2023-07-21T17:15:49.850 INFO:teuthology.lock.ops:Updating [smithi139.front.sepia.ceph.com]: set os type and version on server 2023-07-21T17:15:49.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2023-07-21T17:15:49.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T17:15:54.939 DEBUG:teuthology.orchestra.run.smithi139:> true 2023-07-21T17:15:55.110 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2023-07-21T17:15:55.110 DEBUG:teuthology.orchestra.run.smithi139:> uname -m 2023-07-21T17:15:55.165 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2023-07-21T17:15:55.165 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2023-07-21T17:15:55.219 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="CentOS Stream" 2023-07-21T17:15:55.220 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="9" 2023-07-21T17:15:55.220 INFO:teuthology.orchestra.run.smithi139.stdout:ID="centos" 2023-07-21T17:15:55.220 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="rhel fedora" 2023-07-21T17:15:55.220 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="9" 2023-07-21T17:15:55.220 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el9" 2023-07-21T17:15:55.220 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-21T17:15:55.220 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2023-07-21T17:15:55.221 INFO:teuthology.orchestra.run.smithi139.stdout:LOGO="fedora-logo-icon" 2023-07-21T17:15:55.221 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-21T17:15:55.221 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://centos.org/" 2023-07-21T17:15:55.221 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T17:15:55.221 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-21T17:15:55.221 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-21T17:15:55.222 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2023-07-21T17:16:00.256 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T17:16:00.257 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T17:16:05.355 INFO:teuthology.dispatcher.supervisor:Running job 7346719 2023-07-21T17:16:05.355 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-07-21_15:23:26-fs-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi/7346719 --name yuriw-2023-07-21_15:23:26-fs-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi --description fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}} -- /home/teuthworker/archive/yuriw-2023-07-21_15:23:26-fs-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi/7346719/orig.config.yaml 2023-07-21T17:16:05.359 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-07-21_15:23:26-fs-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi/7346719 2023-07-21T17:16:05.360 INFO:teuthology.dispatcher.supervisor:Job PID: 16823 2023-07-21T17:16:05.360 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-21T17:18:05.460 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T17:18:05.461 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T17:18:10.545 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-07-21T17:18:20.666 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-07-21T17:18:20.667 INFO:teuthology.nuke:Checking targets against current locks 2023-07-21T17:18:39.817 DEBUG:teuthology.nuke:shortname: smithi119 2023-07-21T17:18:39.818 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T17:18:44.849 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-21_15:23:26-fs-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi/7346719', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 17:09:01.754650', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOi8gGUqtZvQTyL83K6H38kc9rarniUNH7LZhivfwHEreXawWtLhl8plVQ8WFnOTNyoDMhHunKmu6z6pjEKypuc='} 2023-07-21T17:18:44.854 DEBUG:teuthology.nuke:shortname: smithi139 2023-07-21T17:18:44.854 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T17:18:44.882 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-21_15:23:26-fs-wip-yuri7-testing-2023-07-20-0727-distro-default-smithi/7346719', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 17:09:01.756813', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH3cCEfOwnks4wO2TTmVo3ZC7faIo92iIQWYk+v0w/fOEsweoo2U5D1/riOo1/Ec6hXqv57WtvwMl2Z0FsmWtlw='} 2023-07-21T17:18:49.921 INFO:teuthology.orchestra.console:Power off smithi119 2023-07-21T17:18:49.922 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T17:18:49.966 INFO:teuthology.orchestra.console:Power off smithi139 2023-07-21T17:18:49.967 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T17:18:49.990 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:18:50.116 INFO:teuthology.orchestra.console:Power off for smithi139 completed 2023-07-21T17:18:50.273 INFO:teuthology.lock.ops:unlocked: smithi139.front.sepia.ceph.com 2023-07-21T17:18:54.947 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:18:59.960 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:19:05.073 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:19:10.186 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:19:15.300 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:19:20.412 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:19:25.529 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:19:30.644 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:19:35.757 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:19:40.869 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:19:45.984 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:19:51.096 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:19:56.207 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:20:01.320 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:20:06.431 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T17:20:07.543 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds 2023-07-21T17:20:07.544 ERROR:teuthology.orchestra.console:Failed to power off smithi119 2023-07-21T17:20:17.609 INFO:teuthology.lock.ops:unlocked: smithi119.front.sepia.ceph.com