2023-10-14T08:32:20.185 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-14T08:32:20.186 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T08:32:20.322 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9953104ca0>, signals=[15]) 2023-10-14T08:32:20.323 INFO:teuthology.lock.ops:Start node 'smithi078.front.sepia.ceph.com' reimaging 2023-10-14T08:32:20.323 INFO:teuthology.lock.ops:Updating [smithi078.front.sepia.ceph.com]: reset os type and version on server 2023-10-14T08:32:20.324 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2023-10-14T08:32:20.349 INFO:teuthology.lock.ops:Start node 'smithi157.front.sepia.ceph.com' reimaging 2023-10-14T08:32:20.350 INFO:teuthology.lock.ops:Updating [smithi157.front.sepia.ceph.com]: reset os type and version on server 2023-10-14T08:32:20.350 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2023-10-14T08:32:20.351 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /host {"name": "smithi078"} 2023-10-14T08:32:20.374 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /host {"name": "smithi157"} 2023-10-14T08:32:20.421 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-10-14T08:32:20.439 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-10-14T08:32:20.468 DEBUG:teuthology.provision.fog.smithi078:Requesting image smithi_rhel_8.6 (ID 5) 2023-10-14T08:32:20.468 DEBUG:teuthology.provision.fog.smithi078:FOG request: PUT /host/183 {"imageID": 5} 2023-10-14T08:32:20.483 DEBUG:teuthology.provision.fog.smithi157:Requesting image smithi_rhel_8.6 (ID 5) 2023-10-14T08:32:20.484 DEBUG:teuthology.provision.fog.smithi157:FOG request: PUT /host/262 {"imageID": 5} 2023-10-14T08:32:20.522 INFO:teuthology.provision.fog.smithi078:Scheduling deploy of rhel 8.6 2023-10-14T08:32:20.523 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-10-14T08:32:20.537 INFO:teuthology.provision.fog.smithi157:Scheduling deploy of rhel 8.6 2023-10-14T08:32:20.537 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-14T08:32:20.588 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /tasktype {"name": "deploy"} 2023-10-14T08:32:20.596 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /tasktype {"name": "deploy"} 2023-10-14T08:32:20.629 DEBUG:teuthology.provision.fog.smithi157:FOG request: POST /host/262/task {"taskTypeID": 1} 2023-10-14T08:32:20.632 DEBUG:teuthology.provision.fog.smithi078:FOG request: POST /host/183/task {"taskTypeID": 1} 2023-10-14T08:32:20.684 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-14T08:32:20.687 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-10-14T08:32:20.792 INFO:teuthology.orchestra.console.smithi078:Power off 2023-10-14T08:32:20.793 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-14T08:32:20.805 INFO:teuthology.orchestra.console.smithi157:Power off 2023-10-14T08:32:20.806 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-14T08:32:20.816 DEBUG:teuthology.orchestra.console.smithi078:power off output: Chassis Power Control: Down/Off 2023-10-14T08:32:20.816 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T08:32:20.825 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2023-10-14T08:32:20.826 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T08:32:20.839 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is on 2023-10-14T08:32:20.847 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-10-14T08:32:24.842 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T08:32:24.849 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T08:32:24.951 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is on 2023-10-14T08:32:24.958 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-10-14T08:32:28.954 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T08:32:28.961 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T08:32:29.063 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is on 2023-10-14T08:32:29.069 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2023-10-14T08:32:29.170 INFO:teuthology.orchestra.console.smithi157:Power off completed 2023-10-14T08:32:29.271 INFO:teuthology.orchestra.console.smithi157:Power on 2023-10-14T08:32:29.271 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-14T08:32:29.294 DEBUG:teuthology.orchestra.console.smithi157:power on output: Chassis Power Control: Up/On 2023-10-14T08:32:29.294 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T08:32:29.315 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2023-10-14T08:32:33.070 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T08:32:33.179 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is off 2023-10-14T08:32:33.280 INFO:teuthology.orchestra.console.smithi078:Power off completed 2023-10-14T08:32:33.316 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T08:32:33.382 INFO:teuthology.orchestra.console.smithi078:Power on 2023-10-14T08:32:33.382 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-14T08:32:33.405 DEBUG:teuthology.orchestra.console.smithi078:power on output: Chassis Power Control: Up/On 2023-10-14T08:32:33.405 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T08:32:33.424 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2023-10-14T08:32:33.426 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is off 2023-10-14T08:32:37.426 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T08:32:37.434 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T08:32:37.536 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2023-10-14T08:32:37.542 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is on 2023-10-14T08:32:37.638 INFO:teuthology.orchestra.console.smithi157:Power on completed 2023-10-14T08:32:37.643 INFO:teuthology.orchestra.console.smithi078:Power on completed 2023-10-14T08:32:37.739 INFO:teuthology.provision.fog.smithi157:Waiting for deploy to finish 2023-10-14T08:32:37.740 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-14T08:32:37.744 INFO:teuthology.provision.fog.smithi078:Waiting for deploy to finish 2023-10-14T08:32:37.744 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-10-14T08:32:52.815 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-14T08:32:52.817 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-10-14T08:33:07.890 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-14T08:33:07.892 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-10-14T08:33:22.966 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-10-14T08:33:22.968 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-14T08:33:38.042 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-10-14T08:33:38.043 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-14T08:33:53.119 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-14T08:33:53.121 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-10-14T08:34:08.194 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-14T08:34:08.226 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-10-14T08:34:23.330 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-10-14T08:34:23.332 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-14T08:34:38.426 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-14T08:34:38.427 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-10-14T08:34:53.547 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-10-14T08:34:53.548 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-14T08:35:08.634 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-10-14T08:35:08.637 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-14T08:35:23.742 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-14T08:35:23.743 DEBUG:teuthology.provision.fog.smithi078:FOG request: GET /task/active None 2023-10-14T08:35:23.826 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T08:35:38.844 DEBUG:teuthology.provision.fog.smithi157:FOG request: GET /task/active None 2023-10-14T08:35:38.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T08:35:42.335 WARNING:teuthology.provision.fog.smithi078:[Errno None] Unable to connect to port 22 on 172.21.15.78 2023-10-14T08:35:48.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T08:35:51.418 WARNING:teuthology.provision.fog.smithi078:[Errno None] Unable to connect to port 22 on 172.21.15.78 2023-10-14T08:35:57.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T08:35:57.434 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-10-14T08:36:00.474 WARNING:teuthology.provision.fog.smithi078:[Errno None] Unable to connect to port 22 on 172.21.15.78 2023-10-14T08:36:03.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T08:36:06.478 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T08:36:06.490 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-10-14T08:36:09.534 WARNING:teuthology.provision.fog.smithi078:[Errno None] Unable to connect to port 22 on 172.21.15.78 2023-10-14T08:36:12.494 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T08:36:15.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T08:36:15.546 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-10-14T08:36:21.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T08:36:24.602 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-10-14T08:36:30.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T08:36:33.658 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-10-14T08:36:39.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T08:36:42.746 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-10-14T08:36:48.750 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T08:36:51.802 WARNING:teuthology.provision.fog.smithi157:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-10-14T08:36:57.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T08:37:15.554 WARNING:teuthology.provision.fog.smithi078:timed out 2023-10-14T08:37:21.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T08:37:21.890 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-14T08:37:21.891 DEBUG:teuthology.orchestra.run.smithi078:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-14T08:37:22.179 INFO:teuthology.provision.fog.smithi078:Node is ready 2023-10-14T08:37:22.179 DEBUG:teuthology.orchestra.run.smithi078:> hostname 2023-10-14T08:37:22.199 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 2023-10-14T08:37:22.199 DEBUG:teuthology.orchestra.run.smithi078:> grep smithi078 /etc/hosts 2023-10-14T08:37:22.259 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78 smithi078.front.sepia.ceph.com smithi078 2023-10-14T08:37:22.260 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname smithi078 2023-10-14T08:37:22.381 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i -e 's/smithi078/smithi078/g' /etc/hosts 2023-10-14T08:37:22.412 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i -e 's/172.21.15.78/172.21.15.78/g' /etc/hosts 2023-10-14T08:37:22.481 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i -e 's/smithi078/smithi078/g' /etc/hostname 2023-10-14T08:37:22.551 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname smithi078 2023-10-14T08:37:22.618 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2023-10-14T08:37:22.675 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="Red Hat Enterprise Linux" 2023-10-14T08:37:22.675 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="8.6 (Ootpa)" 2023-10-14T08:37:22.676 INFO:teuthology.orchestra.run.smithi078.stdout:ID="rhel" 2023-10-14T08:37:22.676 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="fedora" 2023-10-14T08:37:22.676 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="8.6" 2023-10-14T08:37:22.676 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el8" 2023-10-14T08:37:22.676 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-14T08:37:22.677 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2023-10-14T08:37:22.677 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-14T08:37:22.677 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://www.redhat.com/" 2023-10-14T08:37:22.677 INFO:teuthology.orchestra.run.smithi078.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-14T08:37:22.677 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-14T08:37:22.678 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-14T08:37:22.678 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-14T08:37:22.678 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-14T08:37:22.678 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-14T08:37:22.678 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-14T08:37:22.679 INFO:teuthology.provision.fog.smithi078:Deploy complete! 2023-10-14T08:37:22.680 DEBUG:teuthology.parallel:result is None 2023-10-14T08:37:57.846 WARNING:teuthology.provision.fog.smithi157:timed out 2023-10-14T08:38:03.852 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T08:38:04.193 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-14T08:38:04.194 DEBUG:teuthology.orchestra.run.smithi157:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-14T08:38:04.491 INFO:teuthology.provision.fog.smithi157:Node is ready 2023-10-14T08:38:04.491 DEBUG:teuthology.orchestra.run.smithi157:> hostname 2023-10-14T08:38:04.509 INFO:teuthology.orchestra.run.smithi157.stdout:smithi157.front.sepia.ceph.com 2023-10-14T08:38:04.509 DEBUG:teuthology.orchestra.run.smithi157:> grep smithi157.front.sepia.ceph.com /etc/hosts 2023-10-14T08:38:04.567 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.15.157 smithi157.front.sepia.ceph.com smithi157 2023-10-14T08:38:04.568 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname smithi157 2023-10-14T08:38:04.692 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i -e 's/smithi157.front.sepia.ceph.com/smithi157/g' /etc/hosts 2023-10-14T08:38:04.720 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i -e 's/172.21.15.157/172.21.15.157/g' /etc/hosts 2023-10-14T08:38:04.789 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i -e 's/smithi157.front.sepia.ceph.com/smithi157/g' /etc/hostname 2023-10-14T08:38:04.856 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname smithi157 2023-10-14T08:38:04.922 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2023-10-14T08:38:04.980 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Red Hat Enterprise Linux" 2023-10-14T08:38:04.980 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8.6 (Ootpa)" 2023-10-14T08:38:04.980 INFO:teuthology.orchestra.run.smithi157.stdout:ID="rhel" 2023-10-14T08:38:04.981 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="fedora" 2023-10-14T08:38:04.981 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8.6" 2023-10-14T08:38:04.981 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8" 2023-10-14T08:38:04.981 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-14T08:38:04.982 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2023-10-14T08:38:04.982 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-14T08:38:04.982 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.redhat.com/" 2023-10-14T08:38:04.982 INFO:teuthology.orchestra.run.smithi157.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-14T08:38:04.982 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-14T08:38:04.983 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-14T08:38:04.983 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-14T08:38:04.983 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-14T08:38:04.983 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-14T08:38:04.983 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-14T08:38:04.984 INFO:teuthology.provision.fog.smithi157:Deploy complete! 2023-10-14T08:38:04.984 DEBUG:teuthology.parallel:result is None 2023-10-14T08:38:05.137 INFO:teuthology.lock.ops:Checking smithi078.front.sepia.ceph.com 2023-10-14T08:38:05.138 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-14T08:38:05.160 INFO:teuthology.lock.ops:Checking smithi157.front.sepia.ceph.com 2023-10-14T08:38:05.161 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-14T08:38:05.187 INFO:teuthology.lock.ops:Updating [smithi078.front.sepia.ceph.com]: set os type and version on server 2023-10-14T08:38:05.188 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2023-10-14T08:38:05.189 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T08:38:05.256 DEBUG:teuthology.orchestra.run.smithi078:> true 2023-10-14T08:38:05.394 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2023-10-14T08:38:05.394 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2023-10-14T08:38:05.454 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2023-10-14T08:38:05.455 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2023-10-14T08:38:05.514 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="Red Hat Enterprise Linux" 2023-10-14T08:38:05.514 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="8.6 (Ootpa)" 2023-10-14T08:38:05.514 INFO:teuthology.orchestra.run.smithi078.stdout:ID="rhel" 2023-10-14T08:38:05.515 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="fedora" 2023-10-14T08:38:05.515 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="8.6" 2023-10-14T08:38:05.515 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el8" 2023-10-14T08:38:05.515 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-14T08:38:05.516 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2023-10-14T08:38:05.516 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-14T08:38:05.516 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://www.redhat.com/" 2023-10-14T08:38:05.516 INFO:teuthology.orchestra.run.smithi078.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-14T08:38:05.517 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-14T08:38:05.517 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-10-14T08:38:05.517 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-14T08:38:05.517 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-14T08:38:05.518 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-14T08:38:05.518 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-14T08:38:05.519 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2023-10-14T08:38:05.552 INFO:teuthology.lock.ops:Updating [smithi157.front.sepia.ceph.com]: set os type and version on server 2023-10-14T08:38:05.553 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-14T08:38:05.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T08:38:05.619 DEBUG:teuthology.orchestra.run.smithi157:> true 2023-10-14T08:38:05.692 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-10-14T08:38:05.692 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2023-10-14T08:38:05.749 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2023-10-14T08:38:05.749 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2023-10-14T08:38:05.806 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Red Hat Enterprise Linux" 2023-10-14T08:38:05.806 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8.6 (Ootpa)" 2023-10-14T08:38:05.806 INFO:teuthology.orchestra.run.smithi157.stdout:ID="rhel" 2023-10-14T08:38:05.807 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="fedora" 2023-10-14T08:38:05.807 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8.6" 2023-10-14T08:38:05.807 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8" 2023-10-14T08:38:05.807 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-14T08:38:05.807 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2023-10-14T08:38:05.808 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-14T08:38:05.808 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.redhat.com/" 2023-10-14T08:38:05.808 INFO:teuthology.orchestra.run.smithi157.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-14T08:38:05.808 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-14T08:38:05.808 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-10-14T08:38:05.809 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-14T08:38:05.809 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-14T08:38:05.809 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-14T08:38:05.809 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-14T08:38:05.810 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2023-10-14T08:38:05.835 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T08:38:05.941 INFO:teuthology.dispatcher.supervisor:Running job 7425123 2023-10-14T08:38:05.941 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425123 --name rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} -- /home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425123/orig.config.yaml 2023-10-14T08:38:05.944 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425123 2023-10-14T08:38:05.945 INFO:teuthology.dispatcher.supervisor:Job PID: 1543014 2023-10-14T08:38:05.945 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-10-14T08:40:06.042 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T08:42:06.215 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T08:44:06.379 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T08:46:06.531 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T08:48:06.686 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T08:50:06.852 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T08:52:07.015 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T08:54:07.184 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T08:54:07.250 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-10-14T08:54:07.383 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-10-14T08:54:07.384 INFO:teuthology.nuke:Checking targets against current locks 2023-10-14T08:54:07.435 DEBUG:teuthology.nuke:shortname: smithi078 2023-10-14T08:54:07.435 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-14T08:54:07.461 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425123', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-14 08:32:19.377165', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-14T08:54:07.465 DEBUG:teuthology.nuke:shortname: smithi157 2023-10-14T08:54:07.465 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-14T08:54:07.486 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-13_18:04:14-fs-rishabh-2023oct13-testing-default-smithi/7425123', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-14 08:32:19.378961', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-14T08:54:07.523 INFO:teuthology.orchestra.console.smithi078:Power off 2023-10-14T08:54:07.524 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-14T08:54:07.545 DEBUG:teuthology.orchestra.console.smithi078:power off output: Chassis Power Control: Down/Off 2023-10-14T08:54:07.546 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T08:54:07.560 INFO:teuthology.orchestra.console.smithi157:Power off 2023-10-14T08:54:07.561 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-14T08:54:07.571 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is off 2023-10-14T08:54:07.582 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2023-10-14T08:54:07.582 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T08:54:07.603 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2023-10-14T08:54:07.672 INFO:teuthology.orchestra.console.smithi078:Power off completed 2023-10-14T08:54:07.705 INFO:teuthology.orchestra.console.smithi157:Power off completed 2023-10-14T08:54:07.825 INFO:teuthology.lock.ops:unlocked: smithi078.front.sepia.ceph.com 2023-10-14T08:54:07.849 INFO:teuthology.lock.ops:unlocked: smithi157.front.sepia.ceph.com