2023-10-14T18:26:48.295 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-14T18:26:48.297 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T18:26:48.439 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fedbda37670>, signals=[15]) 2023-10-14T18:26:48.441 INFO:teuthology.lock.ops:Start node 'smithi079.front.sepia.ceph.com' reimaging 2023-10-14T18:26:48.441 INFO:teuthology.lock.ops:Updating [smithi079.front.sepia.ceph.com]: reset os type and version on server 2023-10-14T18:26:48.442 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2023-10-14T18:26:48.466 INFO:teuthology.lock.ops:Start node 'smithi144.front.sepia.ceph.com' reimaging 2023-10-14T18:26:48.466 INFO:teuthology.lock.ops:Updating [smithi144.front.sepia.ceph.com]: reset os type and version on server 2023-10-14T18:26:48.466 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2023-10-14T18:26:48.468 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /host {"name": "smithi079"} 2023-10-14T18:26:48.491 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /host {"name": "smithi144"} 2023-10-14T18:26:48.553 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-10-14T18:26:48.564 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-10-14T18:26:48.594 DEBUG:teuthology.provision.fog.smithi079:Requesting image smithi_rhel_8.4 (ID 7) 2023-10-14T18:26:48.595 DEBUG:teuthology.provision.fog.smithi079:FOG request: PUT /host/184 {"imageID": 7} 2023-10-14T18:26:48.608 DEBUG:teuthology.provision.fog.smithi144:Requesting image smithi_rhel_8.4 (ID 7) 2023-10-14T18:26:48.609 DEBUG:teuthology.provision.fog.smithi144:FOG request: PUT /host/249 {"imageID": 7} 2023-10-14T18:26:48.644 INFO:teuthology.provision.fog.smithi079:Scheduling deploy of rhel 8.4 2023-10-14T18:26:48.644 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-10-14T18:26:48.669 INFO:teuthology.provision.fog.smithi144:Scheduling deploy of rhel 8.4 2023-10-14T18:26:48.669 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-10-14T18:26:48.708 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /tasktype {"name": "deploy"} 2023-10-14T18:26:48.723 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /tasktype {"name": "deploy"} 2023-10-14T18:26:48.767 DEBUG:teuthology.provision.fog.smithi144:FOG request: POST /host/249/task {"taskTypeID": 1} 2023-10-14T18:26:48.771 DEBUG:teuthology.provision.fog.smithi079:FOG request: POST /host/184/task {"taskTypeID": 1} 2023-10-14T18:26:48.824 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-10-14T18:26:48.842 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-10-14T18:26:48.924 INFO:teuthology.orchestra.console.smithi144:Power off 2023-10-14T18:26:48.925 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-14T18:26:48.939 INFO:teuthology.orchestra.console.smithi079:Power off 2023-10-14T18:26:48.939 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-14T18:26:48.949 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2023-10-14T18:26:48.950 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T18:26:48.961 DEBUG:teuthology.orchestra.console.smithi079:power off output: Chassis Power Control: Down/Off 2023-10-14T18:26:48.962 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T18:26:48.975 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-10-14T18:26:48.984 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2023-10-14T18:26:52.978 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T18:26:52.986 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T18:26:53.088 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-10-14T18:26:53.095 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2023-10-14T18:26:57.090 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T18:26:57.097 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T18:26:57.199 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-10-14T18:26:57.205 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2023-10-14T18:27:01.202 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T18:27:01.209 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T18:27:01.312 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2023-10-14T18:27:01.318 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off 2023-10-14T18:27:01.414 INFO:teuthology.orchestra.console.smithi144:Power off completed 2023-10-14T18:27:01.419 INFO:teuthology.orchestra.console.smithi079:Power off completed 2023-10-14T18:27:01.515 INFO:teuthology.orchestra.console.smithi144:Power on 2023-10-14T18:27:01.515 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-14T18:27:01.523 INFO:teuthology.orchestra.console.smithi079:Power on 2023-10-14T18:27:01.523 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-14T18:27:01.541 DEBUG:teuthology.orchestra.console.smithi144:power on output: Chassis Power Control: Up/On 2023-10-14T18:27:01.541 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T18:27:01.550 DEBUG:teuthology.orchestra.console.smithi079:power on output: Chassis Power Control: Up/On 2023-10-14T18:27:01.550 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T18:27:01.562 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2023-10-14T18:27:01.571 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off 2023-10-14T18:27:05.566 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T18:27:05.573 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T18:27:05.676 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-10-14T18:27:05.682 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2023-10-14T18:27:05.777 INFO:teuthology.orchestra.console.smithi144:Power on completed 2023-10-14T18:27:05.783 INFO:teuthology.orchestra.console.smithi079:Power on completed 2023-10-14T18:27:05.878 INFO:teuthology.provision.fog.smithi144:Waiting for deploy to finish 2023-10-14T18:27:05.879 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-10-14T18:27:05.884 INFO:teuthology.provision.fog.smithi079:Waiting for deploy to finish 2023-10-14T18:27:05.884 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-10-14T18:27:20.971 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-10-14T18:27:20.973 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-10-14T18:27:36.048 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-10-14T18:27:36.049 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-10-14T18:27:51.126 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-10-14T18:27:51.128 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-10-14T18:28:06.210 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-10-14T18:28:06.212 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-10-14T18:28:21.294 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-10-14T18:28:21.296 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-10-14T18:28:36.382 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-10-14T18:28:36.384 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-10-14T18:28:51.467 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-10-14T18:28:51.469 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-10-14T18:29:06.546 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-10-14T18:29:06.548 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-10-14T18:29:21.622 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-10-14T18:29:21.624 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-10-14T18:29:36.710 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-10-14T18:29:36.712 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-10-14T18:29:51.794 DEBUG:teuthology.provision.fog.smithi079:FOG request: GET /task/active None 2023-10-14T18:29:51.796 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-10-14T18:29:51.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:29:51.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:29:54.942 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-10-14T18:29:54.943 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-10-14T18:30:00.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:30:00.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:30:04.026 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-10-14T18:30:04.027 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-10-14T18:30:10.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:30:10.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:30:13.114 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-10-14T18:30:13.115 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-10-14T18:30:19.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:30:19.124 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:30:22.206 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-10-14T18:30:22.207 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-10-14T18:30:28.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:30:28.216 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:30:31.290 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-10-14T18:30:31.291 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-10-14T18:30:37.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:30:37.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:30:40.350 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-10-14T18:30:46.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:30:49.434 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-10-14T18:30:55.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:31:13.791 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-10-14T18:31:19.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:31:22.881 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-10-14T18:31:28.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:31:31.962 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79 2023-10-14T18:31:37.302 WARNING:teuthology.provision.fog.smithi144:timed out 2023-10-14T18:31:37.965 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:31:38.791 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-14T18:31:38.792 DEBUG:teuthology.orchestra.run.smithi079:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-14T18:31:43.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:31:43.685 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-14T18:31:43.686 DEBUG:teuthology.orchestra.run.smithi144:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-14T18:31:43.919 INFO:teuthology.provision.fog.smithi144:Node is ready 2023-10-14T18:31:43.919 DEBUG:teuthology.orchestra.run.smithi144:> hostname 2023-10-14T18:31:43.975 INFO:teuthology.orchestra.run.smithi144.stdout:smithi144.front.sepia.ceph.com 2023-10-14T18:31:43.975 DEBUG:teuthology.orchestra.run.smithi144:> grep smithi144.front.sepia.ceph.com /etc/hosts 2023-10-14T18:31:44.029 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144 smithi144.front.sepia.ceph.com smithi144 2023-10-14T18:31:44.030 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname smithi144 2023-10-14T18:31:44.143 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i -e 's/smithi144.front.sepia.ceph.com/smithi144/g' /etc/hosts 2023-10-14T18:31:44.170 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i -e 's/172.21.15.144/172.21.15.144/g' /etc/hosts 2023-10-14T18:31:44.234 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i -e 's/smithi144.front.sepia.ceph.com/smithi144/g' /etc/hostname 2023-10-14T18:31:44.298 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname smithi144 2023-10-14T18:31:44.361 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2023-10-14T18:31:44.416 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="Red Hat Enterprise Linux" 2023-10-14T18:31:44.416 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="8.4 (Ootpa)" 2023-10-14T18:31:44.416 INFO:teuthology.orchestra.run.smithi144.stdout:ID="rhel" 2023-10-14T18:31:44.417 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="fedora" 2023-10-14T18:31:44.417 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="8.4" 2023-10-14T18:31:44.417 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el8" 2023-10-14T18:31:44.417 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-14T18:31:44.417 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2023-10-14T18:31:44.418 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-14T18:31:44.418 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://www.redhat.com/" 2023-10-14T18:31:44.418 INFO:teuthology.orchestra.run.smithi144.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-14T18:31:44.418 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-14T18:31:44.418 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-14T18:31:44.419 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-14T18:31:44.419 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-14T18:31:44.419 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-14T18:31:44.419 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-14T18:31:44.420 INFO:teuthology.provision.fog.smithi144:Deploy complete! 2023-10-14T18:31:44.422 DEBUG:teuthology.parallel:result is None 2023-10-14T18:31:54.854 INFO:teuthology.provision.fog.smithi079:Node is ready 2023-10-14T18:31:54.855 DEBUG:teuthology.orchestra.run.smithi079:> hostname 2023-10-14T18:31:54.870 INFO:teuthology.orchestra.run.smithi079.stdout:smithi079.front.sepia.ceph.com 2023-10-14T18:31:54.871 DEBUG:teuthology.orchestra.run.smithi079:> grep smithi079.front.sepia.ceph.com /etc/hosts 2023-10-14T18:31:54.925 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79 smithi079.front.sepia.ceph.com smithi079 2023-10-14T18:31:54.925 DEBUG:teuthology.orchestra.run.smithi079:> sudo hostname smithi079 2023-10-14T18:31:55.099 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/smithi079.front.sepia.ceph.com/smithi079/g' /etc/hosts 2023-10-14T18:31:55.124 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/172.21.15.79/172.21.15.79/g' /etc/hosts 2023-10-14T18:31:55.187 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/smithi079.front.sepia.ceph.com/smithi079/g' /etc/hostname 2023-10-14T18:31:55.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo hostname smithi079 2023-10-14T18:31:55.314 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2023-10-14T18:31:55.367 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="Red Hat Enterprise Linux" 2023-10-14T18:31:55.368 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="8.4 (Ootpa)" 2023-10-14T18:31:55.368 INFO:teuthology.orchestra.run.smithi079.stdout:ID="rhel" 2023-10-14T18:31:55.368 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="fedora" 2023-10-14T18:31:55.368 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="8.4" 2023-10-14T18:31:55.368 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el8" 2023-10-14T18:31:55.369 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-14T18:31:55.369 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2023-10-14T18:31:55.369 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-14T18:31:55.369 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://www.redhat.com/" 2023-10-14T18:31:55.369 INFO:teuthology.orchestra.run.smithi079.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-14T18:31:55.370 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-14T18:31:55.370 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-14T18:31:55.370 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-14T18:31:55.370 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-14T18:31:55.371 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-14T18:31:55.371 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-14T18:31:55.371 INFO:teuthology.provision.fog.smithi079:Deploy complete! 2023-10-14T18:31:55.372 DEBUG:teuthology.parallel:result is None 2023-10-14T18:31:55.524 INFO:teuthology.lock.ops:Checking smithi079.front.sepia.ceph.com 2023-10-14T18:31:55.525 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-14T18:31:55.555 INFO:teuthology.lock.ops:Checking smithi144.front.sepia.ceph.com 2023-10-14T18:31:55.556 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-14T18:31:55.573 INFO:teuthology.lock.ops:Updating [smithi079.front.sepia.ceph.com]: set os type and version on server 2023-10-14T18:31:55.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2023-10-14T18:31:55.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:31:55.642 DEBUG:teuthology.orchestra.run.smithi079:> true 2023-10-14T18:31:55.705 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2023-10-14T18:31:55.705 DEBUG:teuthology.orchestra.run.smithi079:> uname -m 2023-10-14T18:31:55.759 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2023-10-14T18:31:55.759 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2023-10-14T18:31:55.813 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="Red Hat Enterprise Linux" 2023-10-14T18:31:55.813 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="8.4 (Ootpa)" 2023-10-14T18:31:55.813 INFO:teuthology.orchestra.run.smithi079.stdout:ID="rhel" 2023-10-14T18:31:55.814 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="fedora" 2023-10-14T18:31:55.814 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="8.4" 2023-10-14T18:31:55.814 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el8" 2023-10-14T18:31:55.814 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-14T18:31:55.815 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2023-10-14T18:31:55.815 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-14T18:31:55.815 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://www.redhat.com/" 2023-10-14T18:31:55.815 INFO:teuthology.orchestra.run.smithi079.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-14T18:31:55.816 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-14T18:31:55.816 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-10-14T18:31:55.816 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-14T18:31:55.816 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-14T18:31:55.816 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-14T18:31:55.817 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-14T18:31:55.817 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2023-10-14T18:31:55.839 INFO:teuthology.lock.ops:Updating [smithi144.front.sepia.ceph.com]: set os type and version on server 2023-10-14T18:31:55.839 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-10-14T18:31:55.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T18:31:55.948 DEBUG:teuthology.orchestra.run.smithi144:> true 2023-10-14T18:31:56.065 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-10-14T18:31:56.066 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2023-10-14T18:31:56.121 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2023-10-14T18:31:56.121 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2023-10-14T18:31:56.175 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="Red Hat Enterprise Linux" 2023-10-14T18:31:56.176 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="8.4 (Ootpa)" 2023-10-14T18:31:56.176 INFO:teuthology.orchestra.run.smithi144.stdout:ID="rhel" 2023-10-14T18:31:56.176 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="fedora" 2023-10-14T18:31:56.176 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="8.4" 2023-10-14T18:31:56.177 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el8" 2023-10-14T18:31:56.177 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-14T18:31:56.177 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2023-10-14T18:31:56.177 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-14T18:31:56.177 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://www.redhat.com/" 2023-10-14T18:31:56.178 INFO:teuthology.orchestra.run.smithi144.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-14T18:31:56.178 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-14T18:31:56.178 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-14T18:31:56.178 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-14T18:31:56.178 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-14T18:31:56.179 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-14T18:31:56.179 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-14T18:31:56.179 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2023-10-14T18:31:56.205 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T18:31:56.297 INFO:teuthology.dispatcher.supervisor:Running job 7425783 2023-10-14T18:31:56.298 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-10-13_20:07:38-fs-quincy-release-distro-default-smithi/7425783 --name yuriw-2023-10-13_20:07:38-fs-quincy-release-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} -- /home/teuthworker/archive/yuriw-2023-10-13_20:07:38-fs-quincy-release-distro-default-smithi/7425783/orig.config.yaml 2023-10-14T18:31:56.301 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-10-13_20:07:38-fs-quincy-release-distro-default-smithi/7425783 2023-10-14T18:31:56.301 INFO:teuthology.dispatcher.supervisor:Job PID: 479688 2023-10-14T18:31:56.301 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-10-14T18:33:56.400 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T18:35:56.575 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T18:37:56.748 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T18:39:56.919 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T18:41:57.091 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T18:43:57.265 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T18:45:57.431 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T18:47:57.548 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T18:49:57.711 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T18:51:57.875 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T18:53:58.039 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T18:55:58.203 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T18:57:58.371 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T18:57:58.436 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-10-14T18:57:58.558 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-10-14T18:57:58.559 INFO:teuthology.nuke:Checking targets against current locks 2023-10-14T18:57:58.600 DEBUG:teuthology.nuke:shortname: smithi079 2023-10-14T18:57:58.600 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-14T18:57:58.628 DEBUG:teuthology.nuke:shortname: smithi144 2023-10-14T18:57:58.628 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-14T18:57:58.632 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-13_20:07:38-fs-quincy-release-distro-default-smithi/7425783', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-14 18:26:47.461219', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBYk2JnFV1hD/9osJXVTkGSTgA66jIeVDyNBUnusYENk2bNNhg4LLUUYXCAVNS+Lfzt8uNZzM9KaiolF18sa+oo='} 2023-10-14T18:57:58.651 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-13_20:07:38-fs-quincy-release-distro-default-smithi/7425783', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-14 18:26:47.462745', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJrB5DKII1ZhJrgg4sZTw+5WRJyFLlocJrMLwnjOuqNWYLrwG2GQVOyPtDHybcTqAGavr0+uv76+71zljJPXVs8='} 2023-10-14T18:57:58.688 INFO:teuthology.orchestra.console.smithi079:Power off 2023-10-14T18:57:58.688 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-14T18:57:58.709 DEBUG:teuthology.orchestra.console.smithi079:power off output: Chassis Power Control: Down/Off 2023-10-14T18:57:58.710 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T18:57:58.725 INFO:teuthology.orchestra.console.smithi144:Power off 2023-10-14T18:57:58.726 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-14T18:57:58.735 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off 2023-10-14T18:57:58.748 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2023-10-14T18:57:58.748 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T18:57:58.770 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2023-10-14T18:57:58.837 INFO:teuthology.orchestra.console.smithi079:Power off completed 2023-10-14T18:57:58.871 INFO:teuthology.orchestra.console.smithi144:Power off completed 2023-10-14T18:57:58.986 INFO:teuthology.lock.ops:unlocked: smithi079.front.sepia.ceph.com 2023-10-14T18:57:59.027 INFO:teuthology.lock.ops:unlocked: smithi144.front.sepia.ceph.com