2024-04-18T03:29:37.068 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d
2024-04-18T03:29:37.069 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-18T03:29:37.248 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fbac1af1f40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fbac2b52c10>, signals=[15])
2024-04-18T03:29:37.248 INFO:teuthology.lock.ops:Start node 'smithi050.front.sepia.ceph.com' reimaging
2024-04-18T03:29:37.248 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: reset os type and version on server
2024-04-18T03:29:37.248 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server
2024-04-18T03:29:37.267 INFO:teuthology.lock.ops:Start node 'smithi110.front.sepia.ceph.com' reimaging
2024-04-18T03:29:37.267 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: reset os type and version on server
2024-04-18T03:29:37.267 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server
2024-04-18T03:29:37.291 INFO:teuthology.lock.ops:Start node 'smithi164.front.sepia.ceph.com' reimaging
2024-04-18T03:29:37.292 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: reset os type and version on server
2024-04-18T03:29:37.292 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server
2024-04-18T03:29:37.397 DEBUG:teuthology.provision.fog.smithi050:Requesting image smithi_rhel_8.6 (ID 5)
2024-04-18T03:29:37.413 DEBUG:teuthology.provision.fog.smithi110:Requesting image smithi_rhel_8.6 (ID 5)
2024-04-18T03:29:37.419 DEBUG:teuthology.provision.fog.smithi164:Requesting image smithi_rhel_8.6 (ID 5)
2024-04-18T03:29:37.440 INFO:teuthology.provision.fog.smithi050:Scheduling deploy of rhel 8.6
2024-04-18T03:29:37.459 INFO:teuthology.provision.fog.smithi110:Scheduling deploy of rhel 8.6
2024-04-18T03:29:37.462 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of rhel 8.6
2024-04-18T03:29:37.688 INFO:teuthology.orchestra.console.smithi164:Power off
2024-04-18T03:29:37.688 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-18T03:29:37.702 INFO:teuthology.orchestra.console.smithi050:Power off
2024-04-18T03:29:37.702 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-18T03:29:37.717 INFO:teuthology.orchestra.console.smithi110:Power off
2024-04-18T03:29:37.717 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-18T03:29:37.726 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off
2024-04-18T03:29:37.726 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:37.735 DEBUG:teuthology.orchestra.console.smithi050:power off output: Chassis Power Control: Down/Off
2024-04-18T03:29:37.736 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:37.744 DEBUG:teuthology.orchestra.console.smithi110:power off output: Chassis Power Control: Down/Off
2024-04-18T03:29:37.744 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:37.753 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2024-04-18T03:29:37.758 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on
2024-04-18T03:29:37.768 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on
2024-04-18T03:29:41.757 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:41.764 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:41.772 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:41.866 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2024-04-18T03:29:41.874 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on
2024-04-18T03:29:41.881 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on
2024-04-18T03:29:45.870 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:45.877 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:45.883 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:45.979 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2024-04-18T03:29:45.985 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off
2024-04-18T03:29:45.996 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off
2024-04-18T03:29:46.080 INFO:teuthology.orchestra.console.smithi164:Power off completed
2024-04-18T03:29:46.086 INFO:teuthology.orchestra.console.smithi050:Power off completed
2024-04-18T03:29:46.097 INFO:teuthology.orchestra.console.smithi110:Power off completed
2024-04-18T03:29:46.182 INFO:teuthology.orchestra.console.smithi164:Power on
2024-04-18T03:29:46.182 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-18T03:29:46.190 INFO:teuthology.orchestra.console.smithi050:Power on
2024-04-18T03:29:46.190 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-18T03:29:46.198 INFO:teuthology.orchestra.console.smithi110:Power on
2024-04-18T03:29:46.199 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-18T03:29:46.207 DEBUG:teuthology.orchestra.console.smithi164:power on output: Chassis Power Control: Up/On
2024-04-18T03:29:46.207 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:46.217 DEBUG:teuthology.orchestra.console.smithi050:power on output: Chassis Power Control: Up/On
2024-04-18T03:29:46.217 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:46.225 DEBUG:teuthology.orchestra.console.smithi110:power on output: Chassis Power Control: Up/On
2024-04-18T03:29:46.225 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:46.233 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2024-04-18T03:29:46.238 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off
2024-04-18T03:29:46.246 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off
2024-04-18T03:29:50.238 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:50.248 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:50.256 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:50.350 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2024-04-18T03:29:50.358 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off
2024-04-18T03:29:50.365 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off
2024-04-18T03:29:54.354 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:54.362 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:54.368 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T03:29:54.463 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2024-04-18T03:29:54.530 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on
2024-04-18T03:29:54.564 INFO:teuthology.orchestra.console.smithi164:Power on completed
2024-04-18T03:29:54.567 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on
2024-04-18T03:29:54.631 INFO:teuthology.orchestra.console.smithi110:Power on completed
2024-04-18T03:29:54.666 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish
2024-04-18T03:29:54.670 INFO:teuthology.orchestra.console.smithi050:Power on completed
2024-04-18T03:29:54.732 INFO:teuthology.provision.fog.smithi110:Waiting for deploy to finish
2024-04-18T03:29:54.771 INFO:teuthology.provision.fog.smithi050:Waiting for deploy to finish
2024-04-18T03:32:25.524 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:32:25.527 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:32:25.820 ERROR:teuthology.orchestra.connection:Error authenticating with smithi110.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-04-18T03:32:40.582 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:32:43.938 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50
2024-04-18T03:32:49.940 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:32:52.990 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50
2024-04-18T03:32:58.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:32:59.038 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2024-04-18T03:33:02.046 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50
2024-04-18T03:33:05.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:33:08.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:33:08.094 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2024-04-18T03:33:11.106 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50
2024-04-18T03:33:14.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:33:17.108 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:33:17.150 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2024-04-18T03:33:23.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:33:26.210 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2024-04-18T03:33:29.823 WARNING:teuthology.provision.fog.smithi110:timed out
2024-04-18T03:33:32.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:33:35.262 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2024-04-18T03:33:35.824 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:33:41.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:33:44.322 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2024-04-18T03:33:50.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:33:54.334 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110
2024-04-18T03:34:00.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:34:03.166 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-18T03:34:03.167 DEBUG:teuthology.orchestra.run.smithi110:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-18T03:34:17.110 WARNING:teuthology.provision.fog.smithi050:timed out
2024-04-18T03:34:23.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:34:23.387 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-18T03:34:23.387 DEBUG:teuthology.orchestra.run.smithi050:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-18T03:34:23.492 INFO:teuthology.provision.fog.smithi110:Node is ready
2024-04-18T03:34:23.492 DEBUG:teuthology.orchestra.run.smithi110:> hostname
2024-04-18T03:34:23.514 INFO:teuthology.orchestra.run.smithi110.stdout:smithi045
2024-04-18T03:34:23.514 DEBUG:teuthology.orchestra.run.smithi110:> grep smithi045 /etc/hosts
2024-04-18T03:34:23.573 DEBUG:teuthology.orchestra.run:got remote process result: 1
2024-04-18T03:34:23.574 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i -e 's/smithi045/smithi110/g' /etc/hostname
2024-04-18T03:34:23.657 INFO:teuthology.provision.fog.smithi050:Node is ready
2024-04-18T03:34:23.657 DEBUG:teuthology.orchestra.run.smithi050:> hostname
2024-04-18T03:34:23.678 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050.front.sepia.ceph.com
2024-04-18T03:34:23.678 DEBUG:teuthology.orchestra.run.smithi050:> grep smithi050.front.sepia.ceph.com /etc/hosts
2024-04-18T03:34:23.706 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname smithi110
2024-04-18T03:34:23.735 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release
2024-04-18T03:34:23.739 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50 smithi050.front.sepia.ceph.com smithi050
2024-04-18T03:34:23.740 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname smithi050
2024-04-18T03:34:23.793 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="Red Hat Enterprise Linux"
2024-04-18T03:34:23.793 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="8.6 (Ootpa)"
2024-04-18T03:34:23.793 INFO:teuthology.orchestra.run.smithi110.stdout:ID="rhel"
2024-04-18T03:34:23.793 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="fedora"
2024-04-18T03:34:23.793 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="8.6"
2024-04-18T03:34:23.793 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el8"
2024-04-18T03:34:23.793 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2024-04-18T03:34:23.793 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31"
2024-04-18T03:34:23.794 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2024-04-18T03:34:23.794 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://www.redhat.com/"
2024-04-18T03:34:23.794 INFO:teuthology.orchestra.run.smithi110.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2024-04-18T03:34:23.794 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T03:34:23.794 INFO:teuthology.orchestra.run.smithi110.stdout:
2024-04-18T03:34:23.794 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2024-04-18T03:34:23.794 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2024-04-18T03:34:23.794 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2024-04-18T03:34:23.794 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2024-04-18T03:34:23.795 INFO:teuthology.provision.fog.smithi110:Deploy complete!
2024-04-18T03:34:23.796 DEBUG:teuthology.parallel:result is None
2024-04-18T03:34:23.873 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/smithi050.front.sepia.ceph.com/smithi050/g' /etc/hosts
2024-04-18T03:34:23.907 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/172.21.15.50/172.21.15.50/g' /etc/hosts
2024-04-18T03:34:23.983 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/smithi050.front.sepia.ceph.com/smithi050/g' /etc/hostname
2024-04-18T03:34:24.058 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname smithi050
2024-04-18T03:34:24.131 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release
2024-04-18T03:34:24.194 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="Red Hat Enterprise Linux"
2024-04-18T03:34:24.194 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="8.6 (Ootpa)"
2024-04-18T03:34:24.194 INFO:teuthology.orchestra.run.smithi050.stdout:ID="rhel"
2024-04-18T03:34:24.194 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="fedora"
2024-04-18T03:34:24.194 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="8.6"
2024-04-18T03:34:24.194 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el8"
2024-04-18T03:34:24.194 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2024-04-18T03:34:24.195 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31"
2024-04-18T03:34:24.195 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2024-04-18T03:34:24.195 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://www.redhat.com/"
2024-04-18T03:34:24.195 INFO:teuthology.orchestra.run.smithi050.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2024-04-18T03:34:24.195 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T03:34:24.195 INFO:teuthology.orchestra.run.smithi050.stdout:
2024-04-18T03:34:24.195 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2024-04-18T03:34:24.195 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2024-04-18T03:34:24.195 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2024-04-18T03:34:24.195 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2024-04-18T03:34:24.195 INFO:teuthology.provision.fog.smithi050:Deploy complete!
2024-04-18T03:34:24.196 DEBUG:teuthology.parallel:result is None
2024-04-18T03:34:50.326 WARNING:teuthology.provision.fog.smithi164:timed out
2024-04-18T03:34:56.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:34:56.563 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-18T03:34:56.563 DEBUG:teuthology.orchestra.run.smithi164:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-18T03:34:56.861 INFO:teuthology.provision.fog.smithi164:Node is ready
2024-04-18T03:34:56.861 DEBUG:teuthology.orchestra.run.smithi164:> hostname
2024-04-18T03:34:56.880 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164
2024-04-18T03:34:56.880 DEBUG:teuthology.orchestra.run.smithi164:> grep smithi164 /etc/hosts
2024-04-18T03:34:56.938 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164 smithi164.front.sepia.ceph.com smithi164
2024-04-18T03:34:56.939 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164
2024-04-18T03:34:57.067 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164/smithi164/g' /etc/hosts
2024-04-18T03:34:57.097 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/172.21.15.164/172.21.15.164/g' /etc/hosts
2024-04-18T03:34:57.166 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164/smithi164/g' /etc/hostname
2024-04-18T03:34:57.234 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164
2024-04-18T03:34:57.301 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release
2024-04-18T03:34:57.358 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="Red Hat Enterprise Linux"
2024-04-18T03:34:57.358 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="8.6 (Ootpa)"
2024-04-18T03:34:57.358 INFO:teuthology.orchestra.run.smithi164.stdout:ID="rhel"
2024-04-18T03:34:57.358 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="fedora"
2024-04-18T03:34:57.358 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="8.6"
2024-04-18T03:34:57.358 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el8"
2024-04-18T03:34:57.358 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2024-04-18T03:34:57.358 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31"
2024-04-18T03:34:57.358 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2024-04-18T03:34:57.359 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://www.redhat.com/"
2024-04-18T03:34:57.359 INFO:teuthology.orchestra.run.smithi164.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2024-04-18T03:34:57.359 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T03:34:57.359 INFO:teuthology.orchestra.run.smithi164.stdout:
2024-04-18T03:34:57.359 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2024-04-18T03:34:57.359 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2024-04-18T03:34:57.359 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2024-04-18T03:34:57.359 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2024-04-18T03:34:57.359 INFO:teuthology.provision.fog.smithi164:Deploy complete!
2024-04-18T03:34:57.360 DEBUG:teuthology.parallel:result is None
2024-04-18T03:34:57.550 INFO:teuthology.lock.ops:Checking smithi050.front.sepia.ceph.com
2024-04-18T03:34:57.551 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-18T03:34:57.591 INFO:teuthology.lock.ops:Checking smithi110.front.sepia.ceph.com
2024-04-18T03:34:57.591 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-18T03:34:57.615 INFO:teuthology.lock.ops:Checking smithi164.front.sepia.ceph.com
2024-04-18T03:34:57.616 INFO:teuthology.lock.ops:New key found. Updating...
2024-04-18T03:34:57.633 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: set os type and version on server
2024-04-18T03:34:57.634 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com'
2024-04-18T03:34:57.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:34:57.712 DEBUG:teuthology.orchestra.run.smithi050:> true
2024-04-18T03:34:57.855 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com'
2024-04-18T03:34:57.855 DEBUG:teuthology.orchestra.run.smithi050:> uname -m
2024-04-18T03:34:57.915 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64
2024-04-18T03:34:57.916 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release
2024-04-18T03:34:57.977 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="Red Hat Enterprise Linux"
2024-04-18T03:34:57.977 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="8.6 (Ootpa)"
2024-04-18T03:34:57.977 INFO:teuthology.orchestra.run.smithi050.stdout:ID="rhel"
2024-04-18T03:34:57.977 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="fedora"
2024-04-18T03:34:57.977 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="8.6"
2024-04-18T03:34:57.977 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el8"
2024-04-18T03:34:57.977 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2024-04-18T03:34:57.977 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31"
2024-04-18T03:34:57.977 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2024-04-18T03:34:57.977 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://www.redhat.com/"
2024-04-18T03:34:57.978 INFO:teuthology.orchestra.run.smithi050.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2024-04-18T03:34:57.978 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T03:34:57.978 INFO:teuthology.orchestra.run.smithi050.stdout:
2024-04-18T03:34:57.978 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2024-04-18T03:34:57.978 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2024-04-18T03:34:57.978 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2024-04-18T03:34:57.978 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2024-04-18T03:34:57.979 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server
2024-04-18T03:34:57.999 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: set os type and version on server
2024-04-18T03:34:57.999 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com'
2024-04-18T03:34:58.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:34:58.069 DEBUG:teuthology.orchestra.run.smithi110:> true
2024-04-18T03:34:58.208 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com'
2024-04-18T03:34:58.208 DEBUG:teuthology.orchestra.run.smithi110:> uname -m
2024-04-18T03:34:58.266 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64
2024-04-18T03:34:58.266 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release
2024-04-18T03:34:58.324 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="Red Hat Enterprise Linux"
2024-04-18T03:34:58.324 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="8.6 (Ootpa)"
2024-04-18T03:34:58.324 INFO:teuthology.orchestra.run.smithi110.stdout:ID="rhel"
2024-04-18T03:34:58.324 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="fedora"
2024-04-18T03:34:58.324 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="8.6"
2024-04-18T03:34:58.324 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el8"
2024-04-18T03:34:58.324 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2024-04-18T03:34:58.324 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31"
2024-04-18T03:34:58.325 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2024-04-18T03:34:58.325 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://www.redhat.com/"
2024-04-18T03:34:58.325 INFO:teuthology.orchestra.run.smithi110.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2024-04-18T03:34:58.325 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T03:34:58.325 INFO:teuthology.orchestra.run.smithi110.stdout:
2024-04-18T03:34:58.325 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2024-04-18T03:34:58.325 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2024-04-18T03:34:58.325 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2024-04-18T03:34:58.325 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2024-04-18T03:34:58.325 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server
2024-04-18T03:34:58.349 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: set os type and version on server
2024-04-18T03:34:58.349 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com'
2024-04-18T03:34:58.349 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T03:34:58.424 DEBUG:teuthology.orchestra.run.smithi164:> true
2024-04-18T03:34:58.494 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com'
2024-04-18T03:34:58.494 DEBUG:teuthology.orchestra.run.smithi164:> uname -m
2024-04-18T03:34:58.553 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64
2024-04-18T03:34:58.553 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release
2024-04-18T03:34:58.611 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="Red Hat Enterprise Linux"
2024-04-18T03:34:58.611 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="8.6 (Ootpa)"
2024-04-18T03:34:58.611 INFO:teuthology.orchestra.run.smithi164.stdout:ID="rhel"
2024-04-18T03:34:58.611 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="fedora"
2024-04-18T03:34:58.612 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="8.6"
2024-04-18T03:34:58.612 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el8"
2024-04-18T03:34:58.612 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2024-04-18T03:34:58.612 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31"
2024-04-18T03:34:58.612 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2024-04-18T03:34:58.612 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://www.redhat.com/"
2024-04-18T03:34:58.612 INFO:teuthology.orchestra.run.smithi164.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2024-04-18T03:34:58.612 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T03:34:58.612 INFO:teuthology.orchestra.run.smithi164.stdout:
2024-04-18T03:34:58.612 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2024-04-18T03:34:58.612 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2024-04-18T03:34:58.612 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2024-04-18T03:34:58.612 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2024-04-18T03:34:58.613 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server
2024-04-18T03:34:58.632 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-18T03:34:58.735 INFO:teuthology.dispatcher.supervisor:Running job 7661446
2024-04-18T03:34:58.735 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-17_22:24:03-fs-reef-distro-default-smithi/7661446 --name teuthology-2024-04-17_22:24:03-fs-reef-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/postgres}} -- /home/teuthworker/archive/teuthology-2024-04-17_22:24:03-fs-reef-distro-default-smithi/7661446/orig.config.yaml
2024-04-18T03:34:58.739 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2024-04-17_22:24:03-fs-reef-distro-default-smithi/7661446
2024-04-18T03:34:58.739 INFO:teuthology.dispatcher.supervisor:Job PID: 954104
2024-04-18T03:34:58.739 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-04-18T03:36:58.741 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-18T03:38:58.836 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-18T03:40:58.910 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-18T03:42:58.977 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-18T03:44:59.052 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-18T03:46:59.125 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-18T03:48:59.208 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-18T03:48:59.274 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1