2024-01-24T13:01:18.155 INFO:root:teuthology version: 0.0.1.dev251+gc856e0f
2024-01-24T13:01:18.155 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:01:18.306 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fcb06161f40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fcb07107c10>, signals=[15])
2024-01-24T13:01:18.307 INFO:teuthology.lock.ops:Start node 'smithi146.front.sepia.ceph.com' reimaging
2024-01-24T13:01:18.307 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: reset os type and version on server
2024-01-24T13:01:18.307 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2024-01-24T13:01:18.326 INFO:teuthology.lock.ops:Start node 'smithi188.front.sepia.ceph.com' reimaging
2024-01-24T13:01:18.326 INFO:teuthology.lock.ops:Updating [smithi188.front.sepia.ceph.com]: reset os type and version on server
2024-01-24T13:01:18.326 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server
2024-01-24T13:01:18.437 DEBUG:teuthology.provision.fog.smithi146:Requesting image smithi_centos_9.stream (ID 10)
2024-01-24T13:01:18.473 DEBUG:teuthology.provision.fog.smithi188:Requesting image smithi_centos_9.stream (ID 10)
2024-01-24T13:01:18.480 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of centos 9.stream
2024-01-24T13:01:18.517 INFO:teuthology.provision.fog.smithi188:Scheduling deploy of centos 9.stream
2024-01-24T13:01:18.712 INFO:teuthology.orchestra.console.smithi188:Power off
2024-01-24T13:01:18.712 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-24T13:01:18.725 INFO:teuthology.orchestra.console.smithi146:Power off
2024-01-24T13:01:18.726 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-24T13:01:18.735 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off
2024-01-24T13:01:18.735 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-24T13:01:18.746 DEBUG:teuthology.orchestra.console.smithi146:power off output: Chassis Power Control: Down/Off
2024-01-24T13:01:18.746 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-24T13:01:18.756 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on
2024-01-24T13:01:18.768 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on
2024-01-24T13:01:22.760 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-24T13:01:22.769 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-24T13:01:22.870 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on
2024-01-24T13:01:22.877 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on
2024-01-24T13:01:26.875 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-24T13:01:26.882 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-24T13:01:26.985 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on
2024-01-24T13:01:26.991 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on
2024-01-24T13:01:30.990 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-24T13:01:30.997 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-24T13:01:31.100 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off
2024-01-24T13:01:31.105 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off
2024-01-24T13:01:31.202 INFO:teuthology.orchestra.console.smithi188:Power off completed
2024-01-24T13:01:31.206 INFO:teuthology.orchestra.console.smithi146:Power off completed
2024-01-24T13:01:31.303 INFO:teuthology.orchestra.console.smithi188:Power on
2024-01-24T13:01:31.303 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-24T13:01:31.311 INFO:teuthology.orchestra.console.smithi146:Power on
2024-01-24T13:01:31.311 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-24T13:01:31.324 DEBUG:teuthology.orchestra.console.smithi188:power on output: Chassis Power Control: Up/On
2024-01-24T13:01:31.324 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-24T13:01:31.331 DEBUG:teuthology.orchestra.console.smithi146:power on output: Chassis Power Control: Up/On
2024-01-24T13:01:31.331 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-24T13:01:31.346 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off
2024-01-24T13:01:31.353 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off
2024-01-24T13:01:35.348 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-24T13:01:35.356 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-24T13:01:35.458 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on
2024-01-24T13:01:35.464 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on
2024-01-24T13:01:35.559 INFO:teuthology.orchestra.console.smithi188:Power on completed
2024-01-24T13:01:35.565 INFO:teuthology.orchestra.console.smithi146:Power on completed
2024-01-24T13:01:35.660 INFO:teuthology.provision.fog.smithi188:Waiting for deploy to finish
2024-01-24T13:01:35.666 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish
2024-01-24T13:04:21.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-24T13:04:21.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-24T13:04:24.608 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188
2024-01-24T13:04:24.609 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2024-01-24T13:04:30.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-24T13:04:30.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-24T13:04:33.692 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188
2024-01-24T13:04:33.692 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2024-01-24T13:04:39.696 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-24T13:04:39.698 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-24T13:04:42.748 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188
2024-01-24T13:04:42.748 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2024-01-24T13:04:48.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-24T13:04:48.754 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-24T13:04:51.804 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188
2024-01-24T13:04:51.804 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2024-01-24T13:04:57.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-24T13:04:57.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-24T13:05:00.896 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188
2024-01-24T13:05:00.896 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2024-01-24T13:05:06.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-24T13:05:06.900 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-24T13:05:09.980 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2024-01-24T13:05:15.984 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-24T13:06:06.949 WARNING:teuthology.provision.fog.smithi188:timed out
2024-01-24T13:06:12.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-24T13:06:13.292 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-24T13:06:13.293 DEBUG:teuthology.orchestra.run.smithi188:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-24T13:06:13.631 INFO:teuthology.provision.fog.smithi188:Node is ready
2024-01-24T13:06:13.631 DEBUG:teuthology.orchestra.run.smithi188:> hostname
2024-01-24T13:06:13.646 INFO:teuthology.orchestra.run.smithi188.stdout:smithi188.front.sepia.ceph.com
2024-01-24T13:06:13.647 DEBUG:teuthology.orchestra.run.smithi188:> grep smithi188.front.sepia.ceph.com /etc/hosts
2024-01-24T13:06:13.702 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.15.188 smithi188.front.sepia.ceph.com smithi188
2024-01-24T13:06:13.703 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname smithi188
2024-01-24T13:06:13.826 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i -e 's/smithi188.front.sepia.ceph.com/smithi188/g' /etc/hosts
2024-01-24T13:06:13.854 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i -e 's/172.21.15.188/172.21.15.188/g' /etc/hosts
2024-01-24T13:06:13.922 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i -e 's/smithi188.front.sepia.ceph.com/smithi188/g' /etc/hostname
2024-01-24T13:06:13.989 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname smithi188
2024-01-24T13:06:14.055 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release
2024-01-24T13:06:14.109 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream"
2024-01-24T13:06:14.109 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9"
2024-01-24T13:06:14.109 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos"
2024-01-24T13:06:14.110 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora"
2024-01-24T13:06:14.110 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9"
2024-01-24T13:06:14.110 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9"
2024-01-24T13:06:14.110 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-24T13:06:14.110 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31"
2024-01-24T13:06:14.110 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon"
2024-01-24T13:06:14.110 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-24T13:06:14.110 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/"
2024-01-24T13:06:14.110 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-24T13:06:14.110 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-24T13:06:14.110 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-24T13:06:14.111 INFO:teuthology.provision.fog.smithi188:Deploy complete!
2024-01-24T13:06:14.112 DEBUG:teuthology.parallel:result is None
2024-01-24T13:06:15.988 WARNING:teuthology.provision.fog.smithi146:timed out
2024-01-24T13:06:21.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-24T13:06:22.251 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-24T13:06:22.251 DEBUG:teuthology.orchestra.run.smithi146:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-24T13:06:22.576 INFO:teuthology.provision.fog.smithi146:Node is ready
2024-01-24T13:06:22.576 DEBUG:teuthology.orchestra.run.smithi146:> hostname
2024-01-24T13:06:22.590 INFO:teuthology.orchestra.run.smithi146.stdout:smithi146.front.sepia.ceph.com
2024-01-24T13:06:22.591 DEBUG:teuthology.orchestra.run.smithi146:> grep smithi146.front.sepia.ceph.com /etc/hosts
2024-01-24T13:06:22.646 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146 smithi146.front.sepia.ceph.com smithi146
2024-01-24T13:06:22.646 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname smithi146
2024-01-24T13:06:22.753 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/smithi146.front.sepia.ceph.com/smithi146/g' /etc/hosts
2024-01-24T13:06:22.782 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/172.21.15.146/172.21.15.146/g' /etc/hosts
2024-01-24T13:06:22.849 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/smithi146.front.sepia.ceph.com/smithi146/g' /etc/hostname
2024-01-24T13:06:22.918 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname smithi146
2024-01-24T13:06:22.984 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release
2024-01-24T13:06:23.038 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream"
2024-01-24T13:06:23.038 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="9"
2024-01-24T13:06:23.038 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos"
2024-01-24T13:06:23.038 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora"
2024-01-24T13:06:23.038 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="9"
2024-01-24T13:06:23.038 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el9"
2024-01-24T13:06:23.039 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-24T13:06:23.039 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31"
2024-01-24T13:06:23.039 INFO:teuthology.orchestra.run.smithi146.stdout:LOGO="fedora-logo-icon"
2024-01-24T13:06:23.039 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-24T13:06:23.039 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/"
2024-01-24T13:06:23.039 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-24T13:06:23.039 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-24T13:06:23.039 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-24T13:06:23.039 INFO:teuthology.provision.fog.smithi146:Deploy complete!
2024-01-24T13:06:23.040 DEBUG:teuthology.parallel:result is None
2024-01-24T13:06:23.236 INFO:teuthology.lock.ops:Checking smithi146.front.sepia.ceph.com
2024-01-24T13:06:23.236 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-24T13:06:23.287 INFO:teuthology.lock.ops:Checking smithi188.front.sepia.ceph.com
2024-01-24T13:06:23.287 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-24T13:06:23.315 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: set os type and version on server
2024-01-24T13:06:23.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com'
2024-01-24T13:06:23.316 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-24T13:06:23.402 DEBUG:teuthology.orchestra.run.smithi146:> true
2024-01-24T13:06:23.469 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com'
2024-01-24T13:06:23.469 DEBUG:teuthology.orchestra.run.smithi146:> uname -m
2024-01-24T13:06:23.523 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64
2024-01-24T13:06:23.524 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release
2024-01-24T13:06:23.577 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream"
2024-01-24T13:06:23.578 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="9"
2024-01-24T13:06:23.578 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos"
2024-01-24T13:06:23.578 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora"
2024-01-24T13:06:23.578 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="9"
2024-01-24T13:06:23.578 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el9"
2024-01-24T13:06:23.578 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-24T13:06:23.578 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31"
2024-01-24T13:06:23.578 INFO:teuthology.orchestra.run.smithi146.stdout:LOGO="fedora-logo-icon"
2024-01-24T13:06:23.578 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-24T13:06:23.578 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/"
2024-01-24T13:06:23.578 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-24T13:06:23.578 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-24T13:06:23.578 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-24T13:06:23.579 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2024-01-24T13:06:23.614 INFO:teuthology.lock.ops:Updating [smithi188.front.sepia.ceph.com]: set os type and version on server
2024-01-24T13:06:23.615 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com'
2024-01-24T13:06:23.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-24T13:06:23.701 DEBUG:teuthology.orchestra.run.smithi188:> true
2024-01-24T13:06:23.777 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com'
2024-01-24T13:06:23.777 DEBUG:teuthology.orchestra.run.smithi188:> uname -m
2024-01-24T13:06:23.835 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64
2024-01-24T13:06:23.835 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release
2024-01-24T13:06:23.892 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream"
2024-01-24T13:06:23.892 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9"
2024-01-24T13:06:23.893 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos"
2024-01-24T13:06:23.893 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora"
2024-01-24T13:06:23.893 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9"
2024-01-24T13:06:23.893 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9"
2024-01-24T13:06:23.893 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-24T13:06:23.893 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31"
2024-01-24T13:06:23.893 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon"
2024-01-24T13:06:23.893 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-24T13:06:23.893 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/"
2024-01-24T13:06:23.893 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-24T13:06:23.893 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-24T13:06:23.893 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-24T13:06:23.894 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server
2024-01-24T13:06:23.923 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:06:24.012 INFO:teuthology.dispatcher.supervisor:Running job 7530806
2024-01-24T13:06:24.013 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-01-24_12:12:30-fs-wip-vshankar-testing1-reef-2024-01-24-1150-testing-default-smithi/7530806 --name vshankar-2024-01-24_12:12:30-fs-wip-vshankar-testing1-reef-2024-01-24-1150-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} -- /home/teuthworker/archive/vshankar-2024-01-24_12:12:30-fs-wip-vshankar-testing1-reef-2024-01-24-1150-testing-default-smithi/7530806/orig.config.yaml
2024-01-24T13:06:24.016 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-01-24_12:12:30-fs-wip-vshankar-testing1-reef-2024-01-24-1150-testing-default-smithi/7530806
2024-01-24T13:06:24.016 INFO:teuthology.dispatcher.supervisor:Job PID: 2929663
2024-01-24T13:06:24.016 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-01-24T13:08:24.117 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:10:24.281 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:12:24.449 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:14:24.619 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:16:24.788 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:18:24.956 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:20:25.068 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:22:25.229 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:24:25.386 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:26:25.557 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:28:25.724 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:30:25.872 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:32:26.040 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:34:26.216 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:36:26.375 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:38:26.541 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:40:26.716 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:42:26.893 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:44:27.052 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:46:27.211 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:48:27.384 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:50:27.545 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:52:27.704 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:54:27.872 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:56:28.034 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T13:58:28.181 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:00:28.356 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:02:28.521 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:04:28.688 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:06:28.876 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:08:29.045 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:10:29.224 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:12:29.390 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:14:29.561 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:16:29.730 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:18:29.864 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:20:30.029 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:22:30.147 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:24:30.308 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:26:30.486 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:28:30.653 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:30:30.828 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:32:30.996 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:34:31.172 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:36:31.344 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:38:31.520 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:40:31.684 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:42:31.848 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:44:32.016 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:46:32.191 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:48:32.320 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:50:32.488 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:52:32.652 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:54:32.813 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:56:32.996 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T14:58:33.149 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T15:00:33.324 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-24T15:00:33.389 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2024-01-24T15:00:33.570 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2024-01-24T15:00:33.571 INFO:teuthology.nuke:Checking targets against current locks
2024-01-24T15:00:33.621 DEBUG:teuthology.nuke:shortname: smithi146
2024-01-24T15:00:33.622 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-24T15:00:33.643 DEBUG:teuthology.nuke:shortname: smithi188
2024-01-24T15:00:33.643 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-24T15:00:33.647 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-24_12:12:30-fs-wip-vshankar-testing1-reef-2024-01-24-1150-testing-default-smithi/7530806', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-24 13:01:17.049223', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNCaPGPlTq+0i+9mXTXERaTOfSmLchQbqBveIAUw23l/77L5utLWd8NENhuwK+fcErPkiQFspACGgoXEAm4voLM='}
2024-01-24T15:00:33.670 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-24_12:12:30-fs-wip-vshankar-testing1-reef-2024-01-24-1150-testing-default-smithi/7530806', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-24 13:01:17.047347', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKL4NQj+2ZjNZg+kLx6+cQa+IVwxJp/cyYq+52/kWcoYfBJU9DxpUdxDvUifnKb1s/3v1DdJU/Vxa7k/5q8GBic='}
2024-01-24T15:00:33.696 INFO:teuthology.orchestra.console.smithi146:Power off
2024-01-24T15:00:33.697 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-24T15:00:33.719 DEBUG:teuthology.orchestra.console.smithi146:power off output: Chassis Power Control: Down/Off
2024-01-24T15:00:33.719 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-24T15:00:33.732 INFO:teuthology.orchestra.console.smithi188:Power off
2024-01-24T15:00:33.733 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-24T15:00:33.742 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off
2024-01-24T15:00:33.752 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off
2024-01-24T15:00:33.753 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-24T15:00:33.776 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off
2024-01-24T15:00:33.843 INFO:teuthology.orchestra.console.smithi146:Power off completed
2024-01-24T15:00:33.878 INFO:teuthology.orchestra.console.smithi188:Power off completed
2024-01-24T15:00:33.995 INFO:teuthology.lock.ops:unlocked: smithi146.front.sepia.ceph.com
2024-01-24T15:00:34.022 INFO:teuthology.lock.ops:unlocked: smithi188.front.sepia.ceph.com