2023-11-28T08:52:50.220 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5
2023-11-28T08:52:50.220 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T08:52:50.358 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fde0d99cdf0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fde0d6a3dc0>, signals=[15])
2023-11-28T08:52:50.358 INFO:teuthology.lock.ops:Start node 'smithi153.front.sepia.ceph.com' reimaging
2023-11-28T08:52:50.358 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: reset os type and version on server
2023-11-28T08:52:50.358 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server
2023-11-28T08:52:50.381 INFO:teuthology.lock.ops:Start node 'smithi164.front.sepia.ceph.com' reimaging
2023-11-28T08:52:50.381 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: reset os type and version on server
2023-11-28T08:52:50.381 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server
2023-11-28T08:52:50.491 DEBUG:teuthology.provision.fog.smithi153:Requesting image smithi_centos_9.stream (ID 10)
2023-11-28T08:52:50.512 DEBUG:teuthology.provision.fog.smithi164:Requesting image smithi_centos_9.stream (ID 10)
2023-11-28T08:52:50.537 INFO:teuthology.provision.fog.smithi153:Scheduling deploy of centos 9.stream
2023-11-28T08:52:50.559 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of centos 9.stream
2023-11-28T08:52:50.732 INFO:teuthology.orchestra.console.smithi153:Power off
2023-11-28T08:52:50.732 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-28T08:52:50.752 DEBUG:teuthology.orchestra.console.smithi153:power off output: Chassis Power Control: Down/Off
2023-11-28T08:52:50.753 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-28T08:52:50.765 INFO:teuthology.orchestra.console.smithi164:Power off
2023-11-28T08:52:50.765 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-28T08:52:50.774 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on
2023-11-28T08:52:50.786 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off
2023-11-28T08:52:50.786 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-28T08:52:50.806 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2023-11-28T08:52:54.778 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-28T08:52:54.807 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-28T08:52:54.887 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on
2023-11-28T08:52:54.915 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2023-11-28T08:52:58.892 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-28T08:52:58.916 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-28T08:52:59.001 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on
2023-11-28T08:52:59.024 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2023-11-28T08:53:03.006 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-28T08:53:03.025 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-28T08:53:03.116 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off
2023-11-28T08:53:03.619 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2023-11-28T08:53:03.719 INFO:teuthology.orchestra.console.smithi153:Power off completed
2023-11-28T08:53:03.720 INFO:teuthology.orchestra.console.smithi164:Power off completed
2023-11-28T08:53:03.820 INFO:teuthology.orchestra.console.smithi153:Power on
2023-11-28T08:53:03.820 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-28T08:53:03.827 INFO:teuthology.orchestra.console.smithi164:Power on
2023-11-28T08:53:03.828 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-28T08:53:03.841 DEBUG:teuthology.orchestra.console.smithi153:power on output: Chassis Power Control: Up/On
2023-11-28T08:53:03.842 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-28T08:53:03.850 DEBUG:teuthology.orchestra.console.smithi164:power on output: Chassis Power Control: Up/On
2023-11-28T08:53:03.850 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-28T08:53:03.862 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off
2023-11-28T08:53:03.871 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2023-11-28T08:53:07.864 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-28T08:53:07.872 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-28T08:53:07.973 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off
2023-11-28T08:53:07.980 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2023-11-28T08:53:11.974 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-28T08:53:11.981 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-28T08:53:12.083 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on
2023-11-28T08:53:12.090 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2023-11-28T08:53:12.184 INFO:teuthology.orchestra.console.smithi153:Power on completed
2023-11-28T08:53:12.190 INFO:teuthology.orchestra.console.smithi164:Power on completed
2023-11-28T08:53:12.285 INFO:teuthology.provision.fog.smithi153:Waiting for deploy to finish
2023-11-28T08:53:12.291 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish
2023-11-28T08:55:58.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:55:58.289 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:56:01.346 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-11-28T08:56:01.347 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-11-28T08:56:07.349 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:56:07.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:56:10.402 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-11-28T08:56:10.403 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-11-28T08:56:16.406 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:56:16.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:56:19.462 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-11-28T08:56:19.462 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-11-28T08:56:25.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:56:25.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:56:28.546 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-11-28T08:56:28.546 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-11-28T08:56:34.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:56:34.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:56:37.602 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-11-28T08:56:37.602 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-11-28T08:56:43.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:56:43.608 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:56:46.658 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153
2023-11-28T08:56:46.659 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-11-28T08:56:52.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:56:52.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:56:55.746 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2023-11-28T08:57:01.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:57:52.719 WARNING:teuthology.provision.fog.smithi153:timed out
2023-11-28T08:57:58.727 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:57:58.955 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-28T08:57:58.956 DEBUG:teuthology.orchestra.run.smithi153:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-28T08:57:59.254 INFO:teuthology.provision.fog.smithi153:Node is ready
2023-11-28T08:57:59.254 DEBUG:teuthology.orchestra.run.smithi153:> hostname
2023-11-28T08:57:59.273 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153.front.sepia.ceph.com
2023-11-28T08:57:59.273 DEBUG:teuthology.orchestra.run.smithi153:> grep smithi153.front.sepia.ceph.com /etc/hosts
2023-11-28T08:57:59.328 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153 smithi153.front.sepia.ceph.com smithi153
2023-11-28T08:57:59.329 DEBUG:teuthology.orchestra.run.smithi153:> sudo hostname smithi153
2023-11-28T08:57:59.430 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i -e 's/smithi153.front.sepia.ceph.com/smithi153/g' /etc/hosts
2023-11-28T08:57:59.460 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i -e 's/172.21.15.153/172.21.15.153/g' /etc/hosts
2023-11-28T08:57:59.527 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i -e 's/smithi153.front.sepia.ceph.com/smithi153/g' /etc/hostname
2023-11-28T08:57:59.594 DEBUG:teuthology.orchestra.run.smithi153:> sudo hostname smithi153
2023-11-28T08:57:59.661 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release
2023-11-28T08:57:59.715 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream"
2023-11-28T08:57:59.715 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="9"
2023-11-28T08:57:59.716 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos"
2023-11-28T08:57:59.716 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora"
2023-11-28T08:57:59.716 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="9"
2023-11-28T08:57:59.716 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el9"
2023-11-28T08:57:59.716 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 9"
2023-11-28T08:57:59.716 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31"
2023-11-28T08:57:59.716 INFO:teuthology.orchestra.run.smithi153.stdout:LOGO="fedora-logo-icon"
2023-11-28T08:57:59.716 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-11-28T08:57:59.716 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/"
2023-11-28T08:57:59.716 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-28T08:57:59.716 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-11-28T08:57:59.717 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-11-28T08:57:59.717 INFO:teuthology.provision.fog.smithi153:Deploy complete!
2023-11-28T08:57:59.718 DEBUG:teuthology.parallel:result is None
2023-11-28T08:58:01.756 WARNING:teuthology.provision.fog.smithi164:timed out
2023-11-28T08:58:07.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:58:07.989 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-28T08:58:07.990 DEBUG:teuthology.orchestra.run.smithi164:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-28T08:58:08.317 INFO:teuthology.provision.fog.smithi164:Node is ready
2023-11-28T08:58:08.317 DEBUG:teuthology.orchestra.run.smithi164:> hostname
2023-11-28T08:58:08.331 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164.front.sepia.ceph.com
2023-11-28T08:58:08.331 DEBUG:teuthology.orchestra.run.smithi164:> grep smithi164.front.sepia.ceph.com /etc/hosts
2023-11-28T08:58:08.390 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164 smithi164.front.sepia.ceph.com smithi164
2023-11-28T08:58:08.390 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164
2023-11-28T08:58:08.508 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164.front.sepia.ceph.com/smithi164/g' /etc/hosts
2023-11-28T08:58:08.536 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/172.21.15.164/172.21.15.164/g' /etc/hosts
2023-11-28T08:58:08.606 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164.front.sepia.ceph.com/smithi164/g' /etc/hostname
2023-11-28T08:58:08.677 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164
2023-11-28T08:58:08.748 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release
2023-11-28T08:58:08.806 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream"
2023-11-28T08:58:08.806 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9"
2023-11-28T08:58:08.806 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos"
2023-11-28T08:58:08.806 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora"
2023-11-28T08:58:08.806 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9"
2023-11-28T08:58:08.806 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9"
2023-11-28T08:58:08.806 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9"
2023-11-28T08:58:08.806 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31"
2023-11-28T08:58:08.806 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon"
2023-11-28T08:58:08.807 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-11-28T08:58:08.807 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/"
2023-11-28T08:58:08.807 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-28T08:58:08.807 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-11-28T08:58:08.807 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-11-28T08:58:08.807 INFO:teuthology.provision.fog.smithi164:Deploy complete!
2023-11-28T08:58:08.807 DEBUG:teuthology.parallel:result is None
2023-11-28T08:58:08.979 INFO:teuthology.lock.ops:Checking smithi153.front.sepia.ceph.com
2023-11-28T08:58:08.979 INFO:teuthology.lock.ops:New key found. Updating...
2023-11-28T08:58:09.008 INFO:teuthology.lock.ops:Checking smithi164.front.sepia.ceph.com
2023-11-28T08:58:09.008 INFO:teuthology.lock.ops:New key found. Updating...
2023-11-28T08:58:09.034 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: set os type and version on server
2023-11-28T08:58:09.035 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com'
2023-11-28T08:58:09.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:58:09.118 DEBUG:teuthology.orchestra.run.smithi153:> true
2023-11-28T08:58:09.187 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com'
2023-11-28T08:58:09.187 DEBUG:teuthology.orchestra.run.smithi153:> uname -m
2023-11-28T08:58:09.243 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64
2023-11-28T08:58:09.244 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release
2023-11-28T08:58:09.299 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream"
2023-11-28T08:58:09.300 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="9"
2023-11-28T08:58:09.300 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos"
2023-11-28T08:58:09.300 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora"
2023-11-28T08:58:09.300 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="9"
2023-11-28T08:58:09.300 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el9"
2023-11-28T08:58:09.300 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 9"
2023-11-28T08:58:09.300 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31"
2023-11-28T08:58:09.300 INFO:teuthology.orchestra.run.smithi153.stdout:LOGO="fedora-logo-icon"
2023-11-28T08:58:09.300 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-11-28T08:58:09.300 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/"
2023-11-28T08:58:09.300 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-28T08:58:09.300 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-11-28T08:58:09.301 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-11-28T08:58:09.301 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server
2023-11-28T08:58:09.325 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: set os type and version on server
2023-11-28T08:58:09.325 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com'
2023-11-28T08:58:09.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-28T08:58:09.411 DEBUG:teuthology.orchestra.run.smithi164:> true
2023-11-28T08:58:09.476 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com'
2023-11-28T08:58:09.476 DEBUG:teuthology.orchestra.run.smithi164:> uname -m
2023-11-28T08:58:09.534 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64
2023-11-28T08:58:09.534 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release
2023-11-28T08:58:09.592 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream"
2023-11-28T08:58:09.592 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9"
2023-11-28T08:58:09.593 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos"
2023-11-28T08:58:09.593 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora"
2023-11-28T08:58:09.593 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9"
2023-11-28T08:58:09.593 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9"
2023-11-28T08:58:09.593 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9"
2023-11-28T08:58:09.594 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31"
2023-11-28T08:58:09.594 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon"
2023-11-28T08:58:09.594 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-11-28T08:58:09.594 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/"
2023-11-28T08:58:09.594 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-28T08:58:09.595 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-11-28T08:58:09.595 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-11-28T08:58:09.596 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server
2023-11-28T08:58:09.617 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T08:58:09.708 INFO:teuthology.dispatcher.supervisor:Running job 7469858
2023-11-28T08:58:09.708 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-11-28_06:34:49-fs-rishabh-2023nov26-testing-default-smithi/7469858 --name rishabh-2023-11-28_06:34:49-fs-rishabh-2023nov26-testing-default-smithi --description fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/fsstress validater/valgrind} -- /home/teuthworker/archive/rishabh-2023-11-28_06:34:49-fs-rishabh-2023nov26-testing-default-smithi/7469858/orig.config.yaml
2023-11-28T08:58:09.711 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-11-28_06:34:49-fs-rishabh-2023nov26-testing-default-smithi/7469858
2023-11-28T08:58:09.712 INFO:teuthology.dispatcher.supervisor:Job PID: 299363
2023-11-28T08:58:09.712 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-11-28T09:00:09.812 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:02:09.983 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:04:10.175 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:06:10.339 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:08:10.506 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:10:10.671 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:12:10.847 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:14:11.011 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:16:11.183 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:18:11.359 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:20:11.532 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:22:11.699 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:24:11.915 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:26:12.079 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:28:12.247 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:30:12.419 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:32:12.623 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:34:12.795 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:36:12.963 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:38:13.136 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:40:13.303 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:42:13.475 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:44:13.643 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:46:13.810 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:48:13.984 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:50:14.140 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:52:14.307 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:54:14.468 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:56:14.627 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T09:58:14.795 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:00:14.964 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:02:15.131 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:04:15.303 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:06:16.575 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:08:19.832 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:10:20.008 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:12:20.180 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:14:20.347 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:16:20.485 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:18:20.659 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:20:20.839 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:22:21.009 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:24:21.195 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:26:21.379 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:28:21.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:30:21.723 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:32:21.892 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:34:22.055 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:36:22.223 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:38:22.388 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:40:22.553 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:42:22.719 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:44:22.851 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:46:23.015 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:48:23.211 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:50:23.375 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:52:23.559 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:54:23.719 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:56:23.885 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T10:58:24.060 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T11:00:24.219 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T11:02:24.387 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T11:04:24.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T11:06:24.731 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T11:08:24.880 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T11:10:25.056 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T11:12:25.220 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T11:14:25.395 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T11:16:25.563 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T11:18:25.731 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-28T11:18:25.823 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2023-11-28T11:18:26.011 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-11-28T11:18:26.011 INFO:teuthology.nuke:Checking targets against current locks
2023-11-28T11:18:26.056 DEBUG:teuthology.nuke:shortname: smithi153
2023-11-28T11:18:26.056 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-28T11:18:26.074 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-28_06:34:49-fs-rishabh-2023nov26-testing-default-smithi/7469858', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 08:52:49.409026', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPAwea0uHsHOjmjFlfkbcSr2S0YLknKEglBk6OIKxoyMoUqDWSdFI4Uv6YcW9/Y3vBo1E3KrJCYZyHWf/70S8R8='}
2023-11-28T11:18:26.082 DEBUG:teuthology.nuke:shortname: smithi164
2023-11-28T11:18:26.083 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-28T11:18:26.104 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-28_06:34:49-fs-rishabh-2023nov26-testing-default-smithi/7469858', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 08:52:49.407121', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHVm3Yz0bh3kI/Fv9PvFPceLVkevzXH3935CXRp/iimzaMw56jmHa8JiNrR7rMSBdIWI7EiGoCU/zxFzIFZm158='}
2023-11-28T11:18:26.125 INFO:teuthology.orchestra.console.smithi153:Power off
2023-11-28T11:18:26.125 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-28T11:18:26.147 DEBUG:teuthology.orchestra.console.smithi153:power off output: Chassis Power Control: Down/Off
2023-11-28T11:18:26.148 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-28T11:18:26.164 INFO:teuthology.orchestra.console.smithi164:Power off
2023-11-28T11:18:26.164 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-28T11:18:26.172 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off
2023-11-28T11:18:26.186 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off
2023-11-28T11:18:26.186 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-28T11:18:26.207 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2023-11-28T11:18:26.273 INFO:teuthology.orchestra.console.smithi153:Power off completed
2023-11-28T11:18:26.308 INFO:teuthology.orchestra.console.smithi164:Power off completed
2023-11-28T11:18:26.431 INFO:teuthology.lock.ops:unlocked: smithi153.front.sepia.ceph.com
2023-11-28T11:18:26.454 INFO:teuthology.lock.ops:unlocked: smithi164.front.sepia.ceph.com