2023-11-04T01:56:55.002 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-04T01:56:55.004 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T01:56:55.131 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4d383dcca0>, signals=[15]) 2023-11-04T01:56:55.132 INFO:teuthology.lock.ops:Start node 'smithi093.front.sepia.ceph.com' reimaging 2023-11-04T01:56:55.132 INFO:teuthology.lock.ops:Updating [smithi093.front.sepia.ceph.com]: reset os type and version on server 2023-11-04T01:56:55.133 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server 2023-11-04T01:56:55.158 INFO:teuthology.lock.ops:Start node 'smithi114.front.sepia.ceph.com' reimaging 2023-11-04T01:56:55.158 INFO:teuthology.lock.ops:Updating [smithi114.front.sepia.ceph.com]: reset os type and version on server 2023-11-04T01:56:55.158 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2023-11-04T01:56:55.160 DEBUG:teuthology.provision.fog.smithi093:FOG request: GET /host {"name": "smithi093"} 2023-11-04T01:56:55.183 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /host {"name": "smithi114"} 2023-11-04T01:56:55.227 DEBUG:teuthology.provision.fog.smithi093:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-11-04T01:56:55.243 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-11-04T01:56:55.265 DEBUG:teuthology.provision.fog.smithi093:Requesting image smithi_rhel_8.6 (ID 5) 2023-11-04T01:56:55.266 DEBUG:teuthology.provision.fog.smithi093:FOG request: PUT /host/198 {"imageID": 5} 2023-11-04T01:56:55.286 DEBUG:teuthology.provision.fog.smithi114:Requesting image smithi_rhel_8.6 (ID 5) 2023-11-04T01:56:55.287 DEBUG:teuthology.provision.fog.smithi114:FOG request: PUT /host/219 {"imageID": 5} 2023-11-04T01:56:55.323 INFO:teuthology.provision.fog.smithi093:Scheduling deploy of rhel 8.6 2023-11-04T01:56:55.323 DEBUG:teuthology.provision.fog.smithi093:FOG request: GET /task/active None 2023-11-04T01:56:55.333 INFO:teuthology.provision.fog.smithi114:Scheduling deploy of rhel 8.6 2023-11-04T01:56:55.333 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-11-04T01:56:55.381 DEBUG:teuthology.provision.fog.smithi093:FOG request: GET /tasktype {"name": "deploy"} 2023-11-04T01:56:55.389 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /tasktype {"name": "deploy"} 2023-11-04T01:56:55.417 DEBUG:teuthology.provision.fog.smithi093:FOG request: POST /host/198/task {"taskTypeID": 1} 2023-11-04T01:56:55.430 DEBUG:teuthology.provision.fog.smithi114:FOG request: POST /host/219/task {"taskTypeID": 1} 2023-11-04T01:56:55.489 DEBUG:teuthology.provision.fog.smithi093:FOG request: GET /task/active None 2023-11-04T01:56:55.492 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-11-04T01:56:55.580 INFO:teuthology.orchestra.console.smithi093:Power off 2023-11-04T01:56:55.581 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-04T01:56:55.594 INFO:teuthology.orchestra.console.smithi114:Power off 2023-11-04T01:56:55.595 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-04T01:56:55.603 DEBUG:teuthology.orchestra.console.smithi093:power off output: Chassis Power Control: Down/Off 2023-11-04T01:56:55.604 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T01:56:55.617 DEBUG:teuthology.orchestra.console.smithi114:power off output: Chassis Power Control: Down/Off 2023-11-04T01:56:55.618 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T01:56:55.627 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is off 2023-11-04T01:56:55.641 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off 2023-11-04T01:56:55.729 INFO:teuthology.orchestra.console.smithi093:Power off completed 2023-11-04T01:56:55.742 INFO:teuthology.orchestra.console.smithi114:Power off completed 2023-11-04T01:56:55.830 INFO:teuthology.orchestra.console.smithi093:Power on 2023-11-04T01:56:55.831 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-04T01:56:55.843 INFO:teuthology.orchestra.console.smithi114:Power on 2023-11-04T01:56:55.843 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-04T01:56:55.851 DEBUG:teuthology.orchestra.console.smithi093:power on output: Chassis Power Control: Up/On 2023-11-04T01:56:55.851 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T01:56:55.863 DEBUG:teuthology.orchestra.console.smithi114:power on output: Chassis Power Control: Up/On 2023-11-04T01:56:55.864 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T01:56:55.873 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is off 2023-11-04T01:56:55.885 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off 2023-11-04T01:56:59.874 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T01:56:59.886 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T01:56:59.984 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is on 2023-11-04T01:56:59.997 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off 2023-11-04T01:57:00.085 INFO:teuthology.orchestra.console.smithi093:Power on completed 2023-11-04T01:57:00.186 INFO:teuthology.provision.fog.smithi093:Waiting for deploy to finish 2023-11-04T01:57:00.187 DEBUG:teuthology.provision.fog.smithi093:FOG request: GET /task/active None 2023-11-04T01:57:04.001 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T01:57:04.113 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on 2023-11-04T01:57:04.214 INFO:teuthology.orchestra.console.smithi114:Power on completed 2023-11-04T01:57:04.317 INFO:teuthology.provision.fog.smithi114:Waiting for deploy to finish 2023-11-04T01:57:04.318 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-11-04T01:57:15.282 DEBUG:teuthology.provision.fog.smithi093:FOG request: GET /task/active None 2023-11-04T01:57:19.413 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-11-04T01:57:30.366 DEBUG:teuthology.provision.fog.smithi093:FOG request: GET /task/active None 2023-11-04T01:57:34.498 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-11-04T01:57:45.454 DEBUG:teuthology.provision.fog.smithi093:FOG request: GET /task/active None 2023-11-04T01:57:49.574 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-11-04T01:58:00.539 DEBUG:teuthology.provision.fog.smithi093:FOG request: GET /task/active None 2023-11-04T01:58:04.782 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-11-04T01:58:15.618 DEBUG:teuthology.provision.fog.smithi093:FOG request: GET /task/active None 2023-11-04T01:58:19.856 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-11-04T01:58:30.698 DEBUG:teuthology.provision.fog.smithi093:FOG request: GET /task/active None 2023-11-04T01:58:34.934 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-11-04T01:58:45.770 DEBUG:teuthology.provision.fog.smithi093:FOG request: GET /task/active None 2023-11-04T01:58:50.014 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-11-04T01:59:00.858 DEBUG:teuthology.provision.fog.smithi093:FOG request: GET /task/active None 2023-11-04T01:59:05.102 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-11-04T01:59:15.959 DEBUG:teuthology.provision.fog.smithi093:FOG request: GET /task/active None 2023-11-04T01:59:20.216 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-11-04T01:59:31.073 DEBUG:teuthology.provision.fog.smithi093:FOG request: GET /task/active None 2023-11-04T01:59:35.322 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-11-04T01:59:46.159 DEBUG:teuthology.provision.fog.smithi093:FOG request: GET /task/active None 2023-11-04T01:59:46.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T01:59:49.306 WARNING:teuthology.provision.fog.smithi093:[Errno None] Unable to connect to port 22 on 172.21.15.93 2023-11-04T01:59:50.396 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-11-04T01:59:50.476 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T01:59:53.530 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114 2023-11-04T01:59:55.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T01:59:58.366 WARNING:teuthology.provision.fog.smithi093:[Errno None] Unable to connect to port 22 on 172.21.15.93 2023-11-04T01:59:59.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:00:02.586 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114 2023-11-04T02:00:04.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:00:07.454 WARNING:teuthology.provision.fog.smithi093:[Errno None] Unable to connect to port 22 on 172.21.15.93 2023-11-04T02:00:08.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:00:11.642 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114 2023-11-04T02:00:13.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:00:16.538 WARNING:teuthology.provision.fog.smithi093:[Errno None] Unable to connect to port 22 on 172.21.15.93 2023-11-04T02:00:17.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:00:20.702 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114 2023-11-04T02:00:22.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:00:25.594 WARNING:teuthology.provision.fog.smithi093:[Errno None] Unable to connect to port 22 on 172.21.15.93 2023-11-04T02:00:26.704 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:00:29.786 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114 2023-11-04T02:00:31.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:00:34.654 WARNING:teuthology.provision.fog.smithi093:[Errno None] Unable to connect to port 22 on 172.21.15.93 2023-11-04T02:00:35.789 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:00:38.846 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114 2023-11-04T02:00:40.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:00:44.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:00:55.166 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114 2023-11-04T02:01:01.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:01:04.254 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114 2023-11-04T02:01:10.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:01:13.342 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114 2023-11-04T02:01:19.347 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:01:19.702 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-04T02:01:19.703 DEBUG:teuthology.orchestra.run.smithi114:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-04T02:01:20.041 INFO:teuthology.provision.fog.smithi114:Node is ready 2023-11-04T02:01:20.041 DEBUG:teuthology.orchestra.run.smithi114:> hostname 2023-11-04T02:01:20.057 INFO:teuthology.orchestra.run.smithi114.stdout:smithi114.front.sepia.ceph.com 2023-11-04T02:01:20.058 DEBUG:teuthology.orchestra.run.smithi114:> grep smithi114.front.sepia.ceph.com /etc/hosts 2023-11-04T02:01:20.114 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114 smithi114.front.sepia.ceph.com smithi114 2023-11-04T02:01:20.115 DEBUG:teuthology.orchestra.run.smithi114:> sudo hostname smithi114 2023-11-04T02:01:20.248 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i -e 's/smithi114.front.sepia.ceph.com/smithi114/g' /etc/hosts 2023-11-04T02:01:20.276 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i -e 's/172.21.15.114/172.21.15.114/g' /etc/hosts 2023-11-04T02:01:20.344 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i -e 's/smithi114.front.sepia.ceph.com/smithi114/g' /etc/hostname 2023-11-04T02:01:20.410 DEBUG:teuthology.orchestra.run.smithi114:> sudo hostname smithi114 2023-11-04T02:01:20.478 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2023-11-04T02:01:20.535 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="Red Hat Enterprise Linux" 2023-11-04T02:01:20.535 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="8.6 (Ootpa)" 2023-11-04T02:01:20.535 INFO:teuthology.orchestra.run.smithi114.stdout:ID="rhel" 2023-11-04T02:01:20.536 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="fedora" 2023-11-04T02:01:20.536 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="8.6" 2023-11-04T02:01:20.536 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el8" 2023-11-04T02:01:20.536 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-04T02:01:20.536 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2023-11-04T02:01:20.536 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-04T02:01:20.537 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://www.redhat.com/" 2023-11-04T02:01:20.537 INFO:teuthology.orchestra.run.smithi114.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-04T02:01:20.537 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-04T02:01:20.537 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-04T02:01:20.537 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-04T02:01:20.538 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-04T02:01:20.538 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-04T02:01:20.538 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-04T02:01:20.539 INFO:teuthology.provision.fog.smithi114:Deploy complete! 2023-11-04T02:01:20.540 DEBUG:teuthology.parallel:result is None 2023-11-04T02:01:40.680 WARNING:teuthology.provision.fog.smithi093:timed out 2023-11-04T02:01:46.688 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:01:47.255 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-04T02:01:47.256 DEBUG:teuthology.orchestra.run.smithi093:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-04T02:01:47.566 INFO:teuthology.provision.fog.smithi093:Node is ready 2023-11-04T02:01:47.566 DEBUG:teuthology.orchestra.run.smithi093:> hostname 2023-11-04T02:01:47.586 INFO:teuthology.orchestra.run.smithi093.stdout:smithi093.front.sepia.ceph.com 2023-11-04T02:01:47.586 DEBUG:teuthology.orchestra.run.smithi093:> grep smithi093.front.sepia.ceph.com /etc/hosts 2023-11-04T02:01:47.646 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93 smithi093.front.sepia.ceph.com smithi093 2023-11-04T02:01:47.646 DEBUG:teuthology.orchestra.run.smithi093:> sudo hostname smithi093 2023-11-04T02:01:47.762 DEBUG:teuthology.orchestra.run.smithi093:> sudo sed -i -e 's/smithi093.front.sepia.ceph.com/smithi093/g' /etc/hosts 2023-11-04T02:01:47.796 DEBUG:teuthology.orchestra.run.smithi093:> sudo sed -i -e 's/172.21.15.93/172.21.15.93/g' /etc/hosts 2023-11-04T02:01:47.870 DEBUG:teuthology.orchestra.run.smithi093:> sudo sed -i -e 's/smithi093.front.sepia.ceph.com/smithi093/g' /etc/hostname 2023-11-04T02:01:47.943 DEBUG:teuthology.orchestra.run.smithi093:> sudo hostname smithi093 2023-11-04T02:01:48.016 DEBUG:teuthology.orchestra.run.smithi093:> cat /etc/os-release 2023-11-04T02:01:48.076 INFO:teuthology.orchestra.run.smithi093.stdout:NAME="Red Hat Enterprise Linux" 2023-11-04T02:01:48.076 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION="8.6 (Ootpa)" 2023-11-04T02:01:48.076 INFO:teuthology.orchestra.run.smithi093.stdout:ID="rhel" 2023-11-04T02:01:48.076 INFO:teuthology.orchestra.run.smithi093.stdout:ID_LIKE="fedora" 2023-11-04T02:01:48.077 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_ID="8.6" 2023-11-04T02:01:48.077 INFO:teuthology.orchestra.run.smithi093.stdout:PLATFORM_ID="platform:el8" 2023-11-04T02:01:48.077 INFO:teuthology.orchestra.run.smithi093.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-04T02:01:48.077 INFO:teuthology.orchestra.run.smithi093.stdout:ANSI_COLOR="0;31" 2023-11-04T02:01:48.077 INFO:teuthology.orchestra.run.smithi093.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-04T02:01:48.078 INFO:teuthology.orchestra.run.smithi093.stdout:HOME_URL="https://www.redhat.com/" 2023-11-04T02:01:48.078 INFO:teuthology.orchestra.run.smithi093.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-04T02:01:48.078 INFO:teuthology.orchestra.run.smithi093.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-04T02:01:48.078 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-04T02:01:48.079 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-04T02:01:48.079 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-04T02:01:48.079 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-04T02:01:48.079 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-04T02:01:48.079 INFO:teuthology.provision.fog.smithi093:Deploy complete! 2023-11-04T02:01:48.080 DEBUG:teuthology.parallel:result is None 2023-11-04T02:01:48.231 INFO:teuthology.lock.ops:Checking smithi093.front.sepia.ceph.com 2023-11-04T02:01:48.233 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-04T02:01:48.255 INFO:teuthology.lock.ops:Checking smithi114.front.sepia.ceph.com 2023-11-04T02:01:48.256 INFO:teuthology.lock.ops:New key found. Updating... 2023-11-04T02:01:48.281 INFO:teuthology.lock.ops:Updating [smithi093.front.sepia.ceph.com]: set os type and version on server 2023-11-04T02:01:48.282 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-11-04T02:01:48.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:01:48.350 DEBUG:teuthology.orchestra.run.smithi093:> true 2023-11-04T02:01:48.419 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-11-04T02:01:48.419 DEBUG:teuthology.orchestra.run.smithi093:> uname -m 2023-11-04T02:01:48.478 INFO:teuthology.orchestra.run.smithi093.stdout:x86_64 2023-11-04T02:01:48.479 DEBUG:teuthology.orchestra.run.smithi093:> cat /etc/os-release 2023-11-04T02:01:48.538 INFO:teuthology.orchestra.run.smithi093.stdout:NAME="Red Hat Enterprise Linux" 2023-11-04T02:01:48.538 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION="8.6 (Ootpa)" 2023-11-04T02:01:48.539 INFO:teuthology.orchestra.run.smithi093.stdout:ID="rhel" 2023-11-04T02:01:48.539 INFO:teuthology.orchestra.run.smithi093.stdout:ID_LIKE="fedora" 2023-11-04T02:01:48.539 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_ID="8.6" 2023-11-04T02:01:48.539 INFO:teuthology.orchestra.run.smithi093.stdout:PLATFORM_ID="platform:el8" 2023-11-04T02:01:48.539 INFO:teuthology.orchestra.run.smithi093.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-04T02:01:48.540 INFO:teuthology.orchestra.run.smithi093.stdout:ANSI_COLOR="0;31" 2023-11-04T02:01:48.540 INFO:teuthology.orchestra.run.smithi093.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-04T02:01:48.540 INFO:teuthology.orchestra.run.smithi093.stdout:HOME_URL="https://www.redhat.com/" 2023-11-04T02:01:48.540 INFO:teuthology.orchestra.run.smithi093.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-04T02:01:48.541 INFO:teuthology.orchestra.run.smithi093.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-04T02:01:48.541 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-04T02:01:48.541 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-04T02:01:48.541 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-04T02:01:48.541 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-04T02:01:48.542 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-04T02:01:48.543 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server 2023-11-04T02:01:48.563 INFO:teuthology.lock.ops:Updating [smithi114.front.sepia.ceph.com]: set os type and version on server 2023-11-04T02:01:48.564 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2023-11-04T02:01:48.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-04T02:01:48.631 DEBUG:teuthology.orchestra.run.smithi114:> true 2023-11-04T02:01:48.768 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2023-11-04T02:01:48.768 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2023-11-04T02:01:48.826 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2023-11-04T02:01:48.827 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2023-11-04T02:01:48.884 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="Red Hat Enterprise Linux" 2023-11-04T02:01:48.884 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="8.6 (Ootpa)" 2023-11-04T02:01:48.885 INFO:teuthology.orchestra.run.smithi114.stdout:ID="rhel" 2023-11-04T02:01:48.885 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="fedora" 2023-11-04T02:01:48.885 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="8.6" 2023-11-04T02:01:48.885 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el8" 2023-11-04T02:01:48.885 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-04T02:01:48.886 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2023-11-04T02:01:48.886 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-04T02:01:48.886 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://www.redhat.com/" 2023-11-04T02:01:48.886 INFO:teuthology.orchestra.run.smithi114.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-04T02:01:48.886 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-04T02:01:48.887 INFO:teuthology.orchestra.run.smithi114.stdout: 2023-11-04T02:01:48.887 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-04T02:01:48.887 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-04T02:01:48.887 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-04T02:01:48.889 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-04T02:01:48.890 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2023-11-04T02:01:48.914 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:01:49.017 INFO:teuthology.dispatcher.supervisor:Running job 7445738 2023-11-04T02:01:49.018 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-11-03_16:49:22-fs-rishabh-unqiue-fsnames-testing-default-smithi/7445738 --name rishabh-2023-11-03_16:49:22-fs-rishabh-unqiue-fsnames-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} -- /home/teuthworker/archive/rishabh-2023-11-03_16:49:22-fs-rishabh-unqiue-fsnames-testing-default-smithi/7445738/orig.config.yaml 2023-11-04T02:01:49.023 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-11-03_16:49:22-fs-rishabh-unqiue-fsnames-testing-default-smithi/7445738 2023-11-04T02:01:49.024 INFO:teuthology.dispatcher.supervisor:Job PID: 3841087 2023-11-04T02:01:49.024 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-04T02:03:49.122 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:05:49.287 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:07:49.451 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:09:49.619 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:11:49.791 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:13:49.959 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:15:50.119 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:17:50.291 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:19:50.455 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:21:50.624 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-04T02:21:50.688 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-11-04T02:21:50.809 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-04T02:21:50.809 INFO:teuthology.nuke:Checking targets against current locks 2023-11-04T02:21:50.855 DEBUG:teuthology.nuke:shortname: smithi093 2023-11-04T02:21:50.856 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-04T02:21:50.880 DEBUG:teuthology.nuke:shortname: smithi114 2023-11-04T02:21:50.881 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-04T02:21:50.884 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi093.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-03_16:49:22-fs-rishabh-unqiue-fsnames-testing-default-smithi/7445738', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-04 01:56:54.184773', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-04T02:21:50.908 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-03_16:49:22-fs-rishabh-unqiue-fsnames-testing-default-smithi/7445738', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-04 01:56:54.183123', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-04T02:21:50.936 INFO:teuthology.orchestra.console.smithi093:Power off 2023-11-04T02:21:50.937 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-04T02:21:50.960 DEBUG:teuthology.orchestra.console.smithi093:power off output: Chassis Power Control: Down/Off 2023-11-04T02:21:50.961 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T02:21:50.975 INFO:teuthology.orchestra.console.smithi114:Power off 2023-11-04T02:21:50.976 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-04T02:21:50.986 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is off 2023-11-04T02:21:51.001 DEBUG:teuthology.orchestra.console.smithi114:power off output: Chassis Power Control: Down/Off 2023-11-04T02:21:51.001 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-04T02:21:51.024 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off 2023-11-04T02:21:51.088 INFO:teuthology.orchestra.console.smithi093:Power off completed 2023-11-04T02:21:51.126 INFO:teuthology.orchestra.console.smithi114:Power off completed 2023-11-04T02:21:51.245 INFO:teuthology.lock.ops:unlocked: smithi093.front.sepia.ceph.com 2023-11-04T02:21:51.279 INFO:teuthology.lock.ops:unlocked: smithi114.front.sepia.ceph.com