2023-09-28T06:37:40.558 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-28T06:37:40.559 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-28T06:37:40.690 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f87e417b670>, signals=[15]) 2023-09-28T06:37:40.690 INFO:teuthology.lock.ops:Start node 'smithi082.front.sepia.ceph.com' reimaging 2023-09-28T06:37:40.691 INFO:teuthology.lock.ops:Updating [smithi082.front.sepia.ceph.com]: reset os type and version on server 2023-09-28T06:37:40.691 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2023-09-28T06:37:40.710 INFO:teuthology.lock.ops:Start node 'smithi178.front.sepia.ceph.com' reimaging 2023-09-28T06:37:40.710 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: reset os type and version on server 2023-09-28T06:37:40.710 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2023-09-28T06:37:40.711 DEBUG:teuthology.provision.fog.smithi082:FOG request: GET /host {"name": "smithi082"} 2023-09-28T06:37:40.734 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /host {"name": "smithi178"} 2023-09-28T06:37:40.783 DEBUG:teuthology.provision.fog.smithi082:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-09-28T06:37:40.800 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-09-28T06:37:40.834 DEBUG:teuthology.provision.fog.smithi082:Requesting image smithi_rhel_8.4 (ID 7) 2023-09-28T06:37:40.834 DEBUG:teuthology.provision.fog.smithi082:FOG request: PUT /host/187 {"imageID": 7} 2023-09-28T06:37:40.840 DEBUG:teuthology.provision.fog.smithi178:Requesting image smithi_rhel_8.4 (ID 7) 2023-09-28T06:37:40.840 DEBUG:teuthology.provision.fog.smithi178:FOG request: PUT /host/283 {"imageID": 7} 2023-09-28T06:37:40.874 INFO:teuthology.provision.fog.smithi082:Scheduling deploy of rhel 8.4 2023-09-28T06:37:40.874 DEBUG:teuthology.provision.fog.smithi082:FOG request: GET /task/active None 2023-09-28T06:37:40.884 INFO:teuthology.provision.fog.smithi178:Scheduling deploy of rhel 8.4 2023-09-28T06:37:40.884 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-09-28T06:37:40.913 DEBUG:teuthology.provision.fog.smithi082:FOG request: GET /tasktype {"name": "deploy"} 2023-09-28T06:37:40.919 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /tasktype {"name": "deploy"} 2023-09-28T06:37:40.964 DEBUG:teuthology.provision.fog.smithi082:FOG request: POST /host/187/task {"taskTypeID": 1} 2023-09-28T06:37:40.969 DEBUG:teuthology.provision.fog.smithi178:FOG request: POST /host/283/task {"taskTypeID": 1} 2023-09-28T06:37:41.043 DEBUG:teuthology.provision.fog.smithi082:FOG request: GET /task/active None 2023-09-28T06:37:41.049 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-09-28T06:37:41.121 INFO:teuthology.orchestra.console.smithi178:Power off 2023-09-28T06:37:41.122 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-28T06:37:41.134 INFO:teuthology.orchestra.console.smithi082:Power off 2023-09-28T06:37:41.134 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-28T06:37:41.141 DEBUG:teuthology.orchestra.console.smithi178:power off output: Chassis Power Control: Down/Off 2023-09-28T06:37:41.141 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-28T06:37:41.151 DEBUG:teuthology.orchestra.console.smithi082:power off output: Chassis Power Control: Down/Off 2023-09-28T06:37:41.152 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-28T06:37:41.159 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2023-09-28T06:37:41.182 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2023-09-28T06:37:41.261 INFO:teuthology.orchestra.console.smithi178:Power off completed 2023-09-28T06:37:41.283 INFO:teuthology.orchestra.console.smithi082:Power off completed 2023-09-28T06:37:41.361 INFO:teuthology.orchestra.console.smithi178:Power on 2023-09-28T06:37:41.361 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-28T06:37:41.381 DEBUG:teuthology.orchestra.console.smithi178:power on output: Chassis Power Control: Up/On 2023-09-28T06:37:41.382 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-28T06:37:41.388 INFO:teuthology.orchestra.console.smithi082:Power on 2023-09-28T06:37:41.388 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-28T06:37:41.401 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2023-09-28T06:37:41.406 DEBUG:teuthology.orchestra.console.smithi082:power on output: Chassis Power Control: Up/On 2023-09-28T06:37:41.406 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-28T06:37:41.424 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2023-09-28T06:37:45.403 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-28T06:37:45.426 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-28T06:37:45.512 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2023-09-28T06:37:45.533 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2023-09-28T06:37:49.517 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-28T06:37:49.534 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-28T06:37:49.626 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2023-09-28T06:37:49.641 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2023-09-28T06:37:49.727 INFO:teuthology.orchestra.console.smithi178:Power on completed 2023-09-28T06:37:49.742 INFO:teuthology.orchestra.console.smithi082:Power on completed 2023-09-28T06:37:49.828 INFO:teuthology.provision.fog.smithi178:Waiting for deploy to finish 2023-09-28T06:37:49.828 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-09-28T06:37:49.843 INFO:teuthology.provision.fog.smithi082:Waiting for deploy to finish 2023-09-28T06:37:49.843 DEBUG:teuthology.provision.fog.smithi082:FOG request: GET /task/active None 2023-09-28T06:38:04.905 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-09-28T06:38:04.906 DEBUG:teuthology.provision.fog.smithi082:FOG request: GET /task/active None 2023-09-28T06:38:19.974 DEBUG:teuthology.provision.fog.smithi082:FOG request: GET /task/active None 2023-09-28T06:38:19.976 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-09-28T06:38:35.042 DEBUG:teuthology.provision.fog.smithi082:FOG request: GET /task/active None 2023-09-28T06:38:35.043 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-09-28T06:38:50.106 DEBUG:teuthology.provision.fog.smithi082:FOG request: GET /task/active None 2023-09-28T06:38:50.107 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-09-28T06:39:05.178 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-09-28T06:39:05.180 DEBUG:teuthology.provision.fog.smithi082:FOG request: GET /task/active None 2023-09-28T06:39:20.256 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-09-28T06:39:20.258 DEBUG:teuthology.provision.fog.smithi082:FOG request: GET /task/active None 2023-09-28T06:39:35.324 DEBUG:teuthology.provision.fog.smithi082:FOG request: GET /task/active None 2023-09-28T06:39:35.325 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-09-28T06:39:50.390 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-09-28T06:39:50.391 DEBUG:teuthology.provision.fog.smithi082:FOG request: GET /task/active None 2023-09-28T06:40:05.467 DEBUG:teuthology.provision.fog.smithi082:FOG request: GET /task/active None 2023-09-28T06:40:05.469 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-09-28T06:40:20.538 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-09-28T06:40:20.539 DEBUG:teuthology.provision.fog.smithi082:FOG request: GET /task/active None 2023-09-28T06:40:35.622 DEBUG:teuthology.provision.fog.smithi082:FOG request: GET /task/active None 2023-09-28T06:40:35.624 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-09-28T06:40:35.696 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-28T06:40:50.706 DEBUG:teuthology.provision.fog.smithi178:FOG request: GET /task/active None 2023-09-28T06:40:50.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-28T06:40:54.138 WARNING:teuthology.provision.fog.smithi082:[Errno None] Unable to connect to port 22 on 172.21.15.82 2023-09-28T06:41:00.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-28T06:41:03.226 WARNING:teuthology.provision.fog.smithi082:[Errno None] Unable to connect to port 22 on 172.21.15.82 2023-09-28T06:41:09.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-28T06:41:09.242 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-09-28T06:41:12.318 WARNING:teuthology.provision.fog.smithi082:[Errno None] Unable to connect to port 22 on 172.21.15.82 2023-09-28T06:41:15.266 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-28T06:41:18.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-28T06:41:18.330 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-09-28T06:41:21.402 WARNING:teuthology.provision.fog.smithi082:[Errno None] Unable to connect to port 22 on 172.21.15.82 2023-09-28T06:41:24.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-28T06:41:27.386 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-09-28T06:41:27.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-28T06:41:33.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-28T06:41:36.474 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-09-28T06:41:42.481 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-28T06:41:45.562 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-09-28T06:41:51.569 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-28T06:41:54.650 WARNING:teuthology.provision.fog.smithi178:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-09-28T06:42:00.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-28T06:42:27.414 WARNING:teuthology.provision.fog.smithi082:timed out 2023-09-28T06:42:33.421 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-28T06:42:33.796 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-28T06:42:33.797 DEBUG:teuthology.orchestra.run.smithi082:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-28T06:42:34.071 INFO:teuthology.provision.fog.smithi082:Node is ready 2023-09-28T06:42:34.071 DEBUG:teuthology.orchestra.run.smithi082:> hostname 2023-09-28T06:42:34.085 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082 2023-09-28T06:42:34.086 DEBUG:teuthology.orchestra.run.smithi082:> grep smithi082 /etc/hosts 2023-09-28T06:42:34.140 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82 smithi082.front.sepia.ceph.com smithi082 2023-09-28T06:42:34.141 DEBUG:teuthology.orchestra.run.smithi082:> sudo hostname smithi082 2023-09-28T06:42:34.250 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i -e 's/smithi082/smithi082/g' /etc/hosts 2023-09-28T06:42:34.275 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i -e 's/172.21.15.82/172.21.15.82/g' /etc/hosts 2023-09-28T06:42:34.339 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i -e 's/smithi082/smithi082/g' /etc/hostname 2023-09-28T06:42:34.402 DEBUG:teuthology.orchestra.run.smithi082:> sudo hostname smithi082 2023-09-28T06:42:34.465 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2023-09-28T06:42:34.520 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="Red Hat Enterprise Linux" 2023-09-28T06:42:34.520 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="8.4 (Ootpa)" 2023-09-28T06:42:34.520 INFO:teuthology.orchestra.run.smithi082.stdout:ID="rhel" 2023-09-28T06:42:34.520 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="fedora" 2023-09-28T06:42:34.520 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="8.4" 2023-09-28T06:42:34.520 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el8" 2023-09-28T06:42:34.520 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-09-28T06:42:34.520 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2023-09-28T06:42:34.520 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-09-28T06:42:34.521 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://www.redhat.com/" 2023-09-28T06:42:34.521 INFO:teuthology.orchestra.run.smithi082.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-28T06:42:34.521 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-28T06:42:34.521 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-28T06:42:34.521 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-28T06:42:34.521 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-09-28T06:42:34.521 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-28T06:42:34.521 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-09-28T06:42:34.522 INFO:teuthology.provision.fog.smithi082:Deploy complete! 2023-09-28T06:42:34.523 DEBUG:teuthology.parallel:result is None 2023-09-28T06:43:00.682 WARNING:teuthology.provision.fog.smithi178:timed out 2023-09-28T06:43:06.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-28T06:43:07.032 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-28T06:43:07.033 DEBUG:teuthology.orchestra.run.smithi178:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-28T06:43:07.307 INFO:teuthology.provision.fog.smithi178:Node is ready 2023-09-28T06:43:07.307 DEBUG:teuthology.orchestra.run.smithi178:> hostname 2023-09-28T06:43:07.322 INFO:teuthology.orchestra.run.smithi178.stdout:smithi178 2023-09-28T06:43:07.322 DEBUG:teuthology.orchestra.run.smithi178:> grep smithi178 /etc/hosts 2023-09-28T06:43:07.377 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178 smithi178.front.sepia.ceph.com smithi178 2023-09-28T06:43:07.377 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname smithi178 2023-09-28T06:43:07.487 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i -e 's/smithi178/smithi178/g' /etc/hosts 2023-09-28T06:43:07.511 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i -e 's/172.21.15.178/172.21.15.178/g' /etc/hosts 2023-09-28T06:43:07.576 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i -e 's/smithi178/smithi178/g' /etc/hostname 2023-09-28T06:43:07.640 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname smithi178 2023-09-28T06:43:07.703 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2023-09-28T06:43:07.757 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Red Hat Enterprise Linux" 2023-09-28T06:43:07.757 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="8.4 (Ootpa)" 2023-09-28T06:43:07.757 INFO:teuthology.orchestra.run.smithi178.stdout:ID="rhel" 2023-09-28T06:43:07.757 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="fedora" 2023-09-28T06:43:07.757 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="8.4" 2023-09-28T06:43:07.757 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el8" 2023-09-28T06:43:07.757 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-09-28T06:43:07.757 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2023-09-28T06:43:07.757 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-09-28T06:43:07.758 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.redhat.com/" 2023-09-28T06:43:07.758 INFO:teuthology.orchestra.run.smithi178.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-28T06:43:07.758 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-28T06:43:07.758 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-09-28T06:43:07.758 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-28T06:43:07.758 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-09-28T06:43:07.758 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-28T06:43:07.758 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-09-28T06:43:07.758 INFO:teuthology.provision.fog.smithi178:Deploy complete! 2023-09-28T06:43:07.759 DEBUG:teuthology.parallel:result is None 2023-09-28T06:43:08.260 INFO:teuthology.lock.ops:Checking smithi082.front.sepia.ceph.com 2023-09-28T06:43:08.261 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-28T06:43:08.285 INFO:teuthology.lock.ops:Checking smithi178.front.sepia.ceph.com 2023-09-28T06:43:08.285 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-28T06:43:08.312 INFO:teuthology.lock.ops:Updating [smithi082.front.sepia.ceph.com]: set os type and version on server 2023-09-28T06:43:08.312 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2023-09-28T06:43:08.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-28T06:43:08.388 DEBUG:teuthology.orchestra.run.smithi082:> true 2023-09-28T06:43:08.505 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2023-09-28T06:43:08.506 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2023-09-28T06:43:08.567 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2023-09-28T06:43:08.568 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2023-09-28T06:43:08.623 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="Red Hat Enterprise Linux" 2023-09-28T06:43:08.624 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="8.4 (Ootpa)" 2023-09-28T06:43:08.624 INFO:teuthology.orchestra.run.smithi082.stdout:ID="rhel" 2023-09-28T06:43:08.624 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="fedora" 2023-09-28T06:43:08.624 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="8.4" 2023-09-28T06:43:08.624 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el8" 2023-09-28T06:43:08.624 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-09-28T06:43:08.624 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2023-09-28T06:43:08.624 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-09-28T06:43:08.625 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://www.redhat.com/" 2023-09-28T06:43:08.625 INFO:teuthology.orchestra.run.smithi082.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-28T06:43:08.625 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-28T06:43:08.625 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-09-28T06:43:08.625 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-28T06:43:08.625 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-09-28T06:43:08.625 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-28T06:43:08.625 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-09-28T06:43:08.626 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2023-09-28T06:43:08.652 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: set os type and version on server 2023-09-28T06:43:08.652 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-09-28T06:43:08.652 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-28T06:43:08.720 DEBUG:teuthology.orchestra.run.smithi178:> true 2023-09-28T06:43:08.744 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-09-28T06:43:08.744 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2023-09-28T06:43:08.800 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2023-09-28T06:43:08.800 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2023-09-28T06:43:08.856 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Red Hat Enterprise Linux" 2023-09-28T06:43:08.856 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="8.4 (Ootpa)" 2023-09-28T06:43:08.856 INFO:teuthology.orchestra.run.smithi178.stdout:ID="rhel" 2023-09-28T06:43:08.857 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="fedora" 2023-09-28T06:43:08.857 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="8.4" 2023-09-28T06:43:08.857 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el8" 2023-09-28T06:43:08.857 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-09-28T06:43:08.857 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2023-09-28T06:43:08.857 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-09-28T06:43:08.857 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.redhat.com/" 2023-09-28T06:43:08.857 INFO:teuthology.orchestra.run.smithi178.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-28T06:43:08.857 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-28T06:43:08.857 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-09-28T06:43:08.857 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-28T06:43:08.857 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-09-28T06:43:08.858 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-28T06:43:08.858 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-09-28T06:43:08.858 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2023-09-28T06:43:08.877 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-28T06:43:09.046 INFO:teuthology.dispatcher.supervisor:Running job 7403692 2023-09-28T06:43:09.046 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-09-27_19:25:46-fs-wip-yuri2-testing-2023-09-27-0908-quincy-distro-default-smithi/7403692 --name yuriw-2023-09-27_19:25:46-fs-wip-yuri2-testing-2023-09-27-0908-quincy-distro-default-smithi --description fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} -- /home/teuthworker/archive/yuriw-2023-09-27_19:25:46-fs-wip-yuri2-testing-2023-09-27-0908-quincy-distro-default-smithi/7403692/orig.config.yaml 2023-09-28T06:43:09.049 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-09-27_19:25:46-fs-wip-yuri2-testing-2023-09-27-0908-quincy-distro-default-smithi/7403692 2023-09-28T06:43:09.049 INFO:teuthology.dispatcher.supervisor:Job PID: 2294753 2023-09-28T06:43:09.049 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-09-28T06:45:09.096 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-28T06:47:09.267 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-28T06:49:09.427 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-28T06:51:09.559 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-28T06:53:09.719 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-28T06:55:09.899 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-28T06:57:10.075 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-28T06:59:10.314 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-28T07:01:10.491 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-28T07:03:10.816 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-28T07:03:10.896 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-09-28T07:03:11.084 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-09-28T07:03:11.084 INFO:teuthology.nuke:Checking targets against current locks 2023-09-28T07:03:11.133 DEBUG:teuthology.nuke:shortname: smithi082 2023-09-28T07:03:11.133 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-28T07:03:11.154 DEBUG:teuthology.nuke:shortname: smithi178 2023-09-28T07:03:11.154 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-28T07:03:11.160 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-27_19:25:46-fs-wip-yuri2-testing-2023-09-27-0908-quincy-distro-default-smithi/7403692', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-28 06:37:39.758391', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBwLCCZI/IGwHDNKuy7djQmEjVWbDyThOV6/l2cgudxzTT9KqonEJwnfSKoTjaRNuDGhHxwthobYGT+J5lv6BJU='} 2023-09-28T07:03:11.172 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-27_19:25:46-fs-wip-yuri2-testing-2023-09-27-0908-quincy-distro-default-smithi/7403692', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-28 06:37:39.759822', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBENEjkFrBZ2ZsBjyRXEYVEIYX28mMCTm0ED1SBujx6lye+VXerYRjnCtZ+UoN9ZdVhKX/94LRktpWTC7RI6e2rA='} 2023-09-28T07:03:11.208 INFO:teuthology.orchestra.console.smithi082:Power off 2023-09-28T07:03:11.208 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-28T07:03:11.219 INFO:teuthology.orchestra.console.smithi178:Power off 2023-09-28T07:03:11.219 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-28T07:03:11.228 DEBUG:teuthology.orchestra.console.smithi082:power off output: Chassis Power Control: Down/Off 2023-09-28T07:03:11.229 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-28T07:03:11.240 DEBUG:teuthology.orchestra.console.smithi178:power off output: Chassis Power Control: Down/Off 2023-09-28T07:03:11.240 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-28T07:03:11.250 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2023-09-28T07:03:11.259 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2023-09-28T07:03:11.351 INFO:teuthology.orchestra.console.smithi082:Power off completed 2023-09-28T07:03:11.360 INFO:teuthology.orchestra.console.smithi178:Power off completed 2023-09-28T07:03:11.501 INFO:teuthology.lock.ops:unlocked: smithi082.front.sepia.ceph.com 2023-09-28T07:03:11.509 INFO:teuthology.lock.ops:unlocked: smithi178.front.sepia.ceph.com