2023-12-12T09:34:13.217 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e
2023-12-12T09:34:13.218 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T09:34:13.358 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f5ae45b32e0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f5ae54fa940>, signals=[15])
2023-12-12T09:34:13.358 INFO:teuthology.lock.ops:Start node 'smithi017.front.sepia.ceph.com' reimaging
2023-12-12T09:34:13.359 INFO:teuthology.lock.ops:Updating [smithi017.front.sepia.ceph.com]: reset os type and version on server
2023-12-12T09:34:13.359 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server
2023-12-12T09:34:13.376 INFO:teuthology.lock.ops:Start node 'smithi028.front.sepia.ceph.com' reimaging
2023-12-12T09:34:13.377 INFO:teuthology.lock.ops:Updating [smithi028.front.sepia.ceph.com]: reset os type and version on server
2023-12-12T09:34:13.377 INFO:teuthology.lock.ops:Updating smithi028.front.sepia.ceph.com on lock server
2023-12-12T09:34:13.512 DEBUG:teuthology.provision.fog.smithi028:Requesting image smithi_centos_8.stream (ID 3)
2023-12-12T09:34:13.515 DEBUG:teuthology.provision.fog.smithi017:Requesting image smithi_centos_8.stream (ID 3)
2023-12-12T09:34:13.560 INFO:teuthology.provision.fog.smithi028:Scheduling deploy of centos 8.stream
2023-12-12T09:34:13.563 INFO:teuthology.provision.fog.smithi017:Scheduling deploy of centos 8.stream
2023-12-12T09:34:13.799 INFO:teuthology.orchestra.console.smithi017:Power off
2023-12-12T09:34:13.799 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-12T09:34:13.815 INFO:teuthology.orchestra.console.smithi028:Power off
2023-12-12T09:34:13.815 DEBUG:teuthology.orchestra.console.smithi028:pexpect command: ipmitool -H smithi028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-12T09:34:13.824 DEBUG:teuthology.orchestra.console.smithi017:power off output: Chassis Power Control: Down/Off
2023-12-12T09:34:13.824 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T09:34:13.838 DEBUG:teuthology.orchestra.console.smithi028:power off output: Chassis Power Control: Down/Off
2023-12-12T09:34:13.838 DEBUG:teuthology.orchestra.console.smithi028:pexpect command: ipmitool -H smithi028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T09:34:13.848 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on
2023-12-12T09:34:13.862 DEBUG:teuthology.orchestra.console.smithi028:check power output: Chassis Power is on
2023-12-12T09:34:17.850 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T09:34:17.863 DEBUG:teuthology.orchestra.console.smithi028:pexpect command: ipmitool -H smithi028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T09:34:17.964 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on
2023-12-12T09:34:17.975 DEBUG:teuthology.orchestra.console.smithi028:check power output: Chassis Power is on
2023-12-12T09:34:21.966 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T09:34:21.976 DEBUG:teuthology.orchestra.console.smithi028:pexpect command: ipmitool -H smithi028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T09:34:22.076 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is off
2023-12-12T09:34:22.085 DEBUG:teuthology.orchestra.console.smithi028:check power output: Chassis Power is off
2023-12-12T09:34:22.177 INFO:teuthology.orchestra.console.smithi017:Power off completed
2023-12-12T09:34:22.186 INFO:teuthology.orchestra.console.smithi028:Power off completed
2023-12-12T09:34:22.278 INFO:teuthology.orchestra.console.smithi017:Power on
2023-12-12T09:34:22.278 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-12T09:34:22.286 INFO:teuthology.orchestra.console.smithi028:Power on
2023-12-12T09:34:22.287 DEBUG:teuthology.orchestra.console.smithi028:pexpect command: ipmitool -H smithi028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-12-12T09:34:22.301 DEBUG:teuthology.orchestra.console.smithi017:power on output: Chassis Power Control: Up/On
2023-12-12T09:34:22.301 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T09:34:22.311 DEBUG:teuthology.orchestra.console.smithi028:power on output: Chassis Power Control: Up/On
2023-12-12T09:34:22.311 DEBUG:teuthology.orchestra.console.smithi028:pexpect command: ipmitool -H smithi028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T09:34:22.327 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is off
2023-12-12T09:34:22.338 DEBUG:teuthology.orchestra.console.smithi028:check power output: Chassis Power is off
2023-12-12T09:34:26.330 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T09:34:26.339 DEBUG:teuthology.orchestra.console.smithi028:pexpect command: ipmitool -H smithi028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T09:34:26.440 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is off
2023-12-12T09:34:26.448 DEBUG:teuthology.orchestra.console.smithi028:check power output: Chassis Power is off
2023-12-12T09:34:30.442 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T09:34:30.452 DEBUG:teuthology.orchestra.console.smithi028:pexpect command: ipmitool -H smithi028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T09:34:30.554 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on
2023-12-12T09:34:30.564 DEBUG:teuthology.orchestra.console.smithi028:check power output: Chassis Power is on
2023-12-12T09:34:30.655 INFO:teuthology.orchestra.console.smithi017:Power on completed
2023-12-12T09:34:30.665 INFO:teuthology.orchestra.console.smithi028:Power on completed
2023-12-12T09:34:30.757 INFO:teuthology.provision.fog.smithi017:Waiting for deploy to finish
2023-12-12T09:34:30.766 INFO:teuthology.provision.fog.smithi028:Waiting for deploy to finish
2023-12-12T09:37:01.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T09:37:01.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T09:37:04.706 WARNING:teuthology.provision.fog.smithi017:[Errno None] Unable to connect to port 22 on 172.21.15.17
2023-12-12T09:37:04.707 WARNING:teuthology.provision.fog.smithi028:[Errno None] Unable to connect to port 22 on 172.21.15.28
2023-12-12T09:37:10.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T09:37:10.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T09:37:13.762 WARNING:teuthology.provision.fog.smithi017:[Errno None] Unable to connect to port 22 on 172.21.15.17
2023-12-12T09:37:13.795 WARNING:teuthology.provision.fog.smithi028:[Errno None] Unable to connect to port 22 on 172.21.15.28
2023-12-12T09:37:19.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T09:37:19.797 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T09:37:22.819 WARNING:teuthology.provision.fog.smithi017:[Errno None] Unable to connect to port 22 on 172.21.15.17
2023-12-12T09:37:22.854 WARNING:teuthology.provision.fog.smithi028:[Errno None] Unable to connect to port 22 on 172.21.15.28
2023-12-12T09:37:28.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T09:37:28.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T09:37:31.910 WARNING:teuthology.provision.fog.smithi017:[Errno None] Unable to connect to port 22 on 172.21.15.17
2023-12-12T09:37:31.910 WARNING:teuthology.provision.fog.smithi028:[Errno None] Unable to connect to port 22 on 172.21.15.28
2023-12-12T09:37:37.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T09:37:37.916 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T09:37:40.994 WARNING:teuthology.provision.fog.smithi017:[Errno None] Unable to connect to port 22 on 172.21.15.17
2023-12-12T09:37:46.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T09:38:37.968 WARNING:teuthology.provision.fog.smithi028:timed out
2023-12-12T09:38:43.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T09:38:44.228 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-12T09:38:44.229 DEBUG:teuthology.orchestra.run.smithi028:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-12T09:38:44.479 INFO:teuthology.provision.fog.smithi028:Node is ready
2023-12-12T09:38:44.479 DEBUG:teuthology.orchestra.run.smithi028:> hostname
2023-12-12T09:38:44.499 INFO:teuthology.orchestra.run.smithi028.stdout:smithi028.front.sepia.ceph.com
2023-12-12T09:38:44.499 DEBUG:teuthology.orchestra.run.smithi028:> grep smithi028.front.sepia.ceph.com /etc/hosts
2023-12-12T09:38:44.559 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28 smithi028.front.sepia.ceph.com smithi028
2023-12-12T09:38:44.559 DEBUG:teuthology.orchestra.run.smithi028:> sudo hostname smithi028
2023-12-12T09:38:44.659 DEBUG:teuthology.orchestra.run.smithi028:> sudo sed -i -e 's/smithi028.front.sepia.ceph.com/smithi028/g' /etc/hosts
2023-12-12T09:38:44.690 DEBUG:teuthology.orchestra.run.smithi028:> sudo sed -i -e 's/172.21.15.28/172.21.15.28/g' /etc/hosts
2023-12-12T09:38:44.760 DEBUG:teuthology.orchestra.run.smithi028:> sudo sed -i -e 's/smithi028.front.sepia.ceph.com/smithi028/g' /etc/hostname
2023-12-12T09:38:44.831 DEBUG:teuthology.orchestra.run.smithi028:> sudo hostname smithi028
2023-12-12T09:38:44.899 DEBUG:teuthology.orchestra.run.smithi028:> cat /etc/os-release
2023-12-12T09:38:44.958 INFO:teuthology.orchestra.run.smithi028.stdout:NAME="CentOS Stream"
2023-12-12T09:38:44.958 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION="8"
2023-12-12T09:38:44.958 INFO:teuthology.orchestra.run.smithi028.stdout:ID="centos"
2023-12-12T09:38:44.958 INFO:teuthology.orchestra.run.smithi028.stdout:ID_LIKE="rhel fedora"
2023-12-12T09:38:44.958 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION_ID="8"
2023-12-12T09:38:44.958 INFO:teuthology.orchestra.run.smithi028.stdout:PLATFORM_ID="platform:el8"
2023-12-12T09:38:44.958 INFO:teuthology.orchestra.run.smithi028.stdout:PRETTY_NAME="CentOS Stream 8"
2023-12-12T09:38:44.958 INFO:teuthology.orchestra.run.smithi028.stdout:ANSI_COLOR="0;31"
2023-12-12T09:38:44.959 INFO:teuthology.orchestra.run.smithi028.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-12-12T09:38:44.959 INFO:teuthology.orchestra.run.smithi028.stdout:HOME_URL="https://centos.org/"
2023-12-12T09:38:44.959 INFO:teuthology.orchestra.run.smithi028.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-12T09:38:44.959 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-12T09:38:44.959 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-12-12T09:38:44.962 INFO:teuthology.provision.fog.smithi028:Deploy complete!
2023-12-12T09:38:44.963 DEBUG:teuthology.parallel:result is None
2023-12-12T09:38:47.004 WARNING:teuthology.provision.fog.smithi017:timed out
2023-12-12T09:38:53.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T09:38:53.226 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-12-12T09:38:53.226 DEBUG:teuthology.orchestra.run.smithi017:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-12-12T09:38:53.494 INFO:teuthology.provision.fog.smithi017:Node is ready
2023-12-12T09:38:53.494 DEBUG:teuthology.orchestra.run.smithi017:> hostname
2023-12-12T09:38:53.514 INFO:teuthology.orchestra.run.smithi017.stdout:smithi017.front.sepia.ceph.com
2023-12-12T09:38:53.515 DEBUG:teuthology.orchestra.run.smithi017:> grep smithi017.front.sepia.ceph.com /etc/hosts
2023-12-12T09:38:53.577 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17 smithi017.front.sepia.ceph.com smithi017
2023-12-12T09:38:53.578 DEBUG:teuthology.orchestra.run.smithi017:> sudo hostname smithi017
2023-12-12T09:38:53.678 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i -e 's/smithi017.front.sepia.ceph.com/smithi017/g' /etc/hosts
2023-12-12T09:38:53.709 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i -e 's/172.21.15.17/172.21.15.17/g' /etc/hosts
2023-12-12T09:38:53.779 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i -e 's/smithi017.front.sepia.ceph.com/smithi017/g' /etc/hostname
2023-12-12T09:38:53.849 DEBUG:teuthology.orchestra.run.smithi017:> sudo hostname smithi017
2023-12-12T09:38:53.918 DEBUG:teuthology.orchestra.run.smithi017:> cat /etc/os-release
2023-12-12T09:38:53.977 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="CentOS Stream"
2023-12-12T09:38:53.977 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="8"
2023-12-12T09:38:53.977 INFO:teuthology.orchestra.run.smithi017.stdout:ID="centos"
2023-12-12T09:38:53.978 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE="rhel fedora"
2023-12-12T09:38:53.978 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="8"
2023-12-12T09:38:53.978 INFO:teuthology.orchestra.run.smithi017.stdout:PLATFORM_ID="platform:el8"
2023-12-12T09:38:53.978 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="CentOS Stream 8"
2023-12-12T09:38:53.978 INFO:teuthology.orchestra.run.smithi017.stdout:ANSI_COLOR="0;31"
2023-12-12T09:38:53.978 INFO:teuthology.orchestra.run.smithi017.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-12-12T09:38:53.978 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://centos.org/"
2023-12-12T09:38:53.978 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-12T09:38:53.978 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-12T09:38:53.978 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-12-12T09:38:53.979 INFO:teuthology.provision.fog.smithi017:Deploy complete!
2023-12-12T09:38:53.979 DEBUG:teuthology.parallel:result is None
2023-12-12T09:38:54.136 INFO:teuthology.lock.ops:Checking smithi017.front.sepia.ceph.com
2023-12-12T09:38:54.137 INFO:teuthology.lock.ops:New key found. Updating...
2023-12-12T09:38:54.186 INFO:teuthology.lock.ops:Checking smithi028.front.sepia.ceph.com
2023-12-12T09:38:54.186 INFO:teuthology.lock.ops:New key found. Updating...
2023-12-12T09:38:54.212 INFO:teuthology.lock.ops:Updating [smithi017.front.sepia.ceph.com]: set os type and version on server
2023-12-12T09:38:54.213 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com'
2023-12-12T09:38:54.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T09:38:54.281 DEBUG:teuthology.orchestra.run.smithi017:> true
2023-12-12T09:38:54.353 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com'
2023-12-12T09:38:54.353 DEBUG:teuthology.orchestra.run.smithi017:> uname -m
2023-12-12T09:38:54.411 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64
2023-12-12T09:38:54.412 DEBUG:teuthology.orchestra.run.smithi017:> cat /etc/os-release
2023-12-12T09:38:54.471 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="CentOS Stream"
2023-12-12T09:38:54.471 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="8"
2023-12-12T09:38:54.471 INFO:teuthology.orchestra.run.smithi017.stdout:ID="centos"
2023-12-12T09:38:54.471 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE="rhel fedora"
2023-12-12T09:38:54.471 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="8"
2023-12-12T09:38:54.471 INFO:teuthology.orchestra.run.smithi017.stdout:PLATFORM_ID="platform:el8"
2023-12-12T09:38:54.471 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="CentOS Stream 8"
2023-12-12T09:38:54.471 INFO:teuthology.orchestra.run.smithi017.stdout:ANSI_COLOR="0;31"
2023-12-12T09:38:54.471 INFO:teuthology.orchestra.run.smithi017.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-12-12T09:38:54.471 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://centos.org/"
2023-12-12T09:38:54.471 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-12T09:38:54.471 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-12T09:38:54.472 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-12-12T09:38:54.472 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server
2023-12-12T09:38:54.494 INFO:teuthology.lock.ops:Updating [smithi028.front.sepia.ceph.com]: set os type and version on server
2023-12-12T09:38:54.494 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com'
2023-12-12T09:38:54.495 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-12-12T09:38:54.564 DEBUG:teuthology.orchestra.run.smithi028:> true
2023-12-12T09:38:54.640 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi028.front.sepia.ceph.com'
2023-12-12T09:38:54.640 DEBUG:teuthology.orchestra.run.smithi028:> uname -m
2023-12-12T09:38:54.703 INFO:teuthology.orchestra.run.smithi028.stdout:x86_64
2023-12-12T09:38:54.703 DEBUG:teuthology.orchestra.run.smithi028:> cat /etc/os-release
2023-12-12T09:38:54.765 INFO:teuthology.orchestra.run.smithi028.stdout:NAME="CentOS Stream"
2023-12-12T09:38:54.765 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION="8"
2023-12-12T09:38:54.766 INFO:teuthology.orchestra.run.smithi028.stdout:ID="centos"
2023-12-12T09:38:54.766 INFO:teuthology.orchestra.run.smithi028.stdout:ID_LIKE="rhel fedora"
2023-12-12T09:38:54.766 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION_ID="8"
2023-12-12T09:38:54.766 INFO:teuthology.orchestra.run.smithi028.stdout:PLATFORM_ID="platform:el8"
2023-12-12T09:38:54.766 INFO:teuthology.orchestra.run.smithi028.stdout:PRETTY_NAME="CentOS Stream 8"
2023-12-12T09:38:54.766 INFO:teuthology.orchestra.run.smithi028.stdout:ANSI_COLOR="0;31"
2023-12-12T09:38:54.766 INFO:teuthology.orchestra.run.smithi028.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-12-12T09:38:54.766 INFO:teuthology.orchestra.run.smithi028.stdout:HOME_URL="https://centos.org/"
2023-12-12T09:38:54.767 INFO:teuthology.orchestra.run.smithi028.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-12-12T09:38:54.767 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-12-12T09:38:54.767 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-12-12T09:38:54.767 INFO:teuthology.lock.ops:Updating smithi028.front.sepia.ceph.com on lock server
2023-12-12T09:38:54.785 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T09:38:54.880 INFO:teuthology.dispatcher.supervisor:Running job 7488226
2023-12-12T09:38:54.880 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/bin/teuthology -v --owner mchangir --archive /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488226 --name mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488226/orig.config.yaml
2023-12-12T09:38:54.888 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488226
2023-12-12T09:38:54.888 INFO:teuthology.dispatcher.supervisor:Job PID: 2703796
2023-12-12T09:38:54.889 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-12-12T09:40:54.987 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T09:42:55.151 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T09:44:55.319 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T09:46:55.497 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T09:48:55.661 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T09:50:55.831 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T09:52:56.031 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T09:54:56.203 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T09:56:56.375 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T09:58:56.550 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T10:00:56.724 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T10:02:56.899 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T10:04:57.063 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T10:06:57.234 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T10:08:57.415 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T10:10:57.595 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T10:12:57.763 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T10:14:57.926 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T10:16:58.095 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T10:18:58.263 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T10:20:58.431 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T10:22:58.606 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-12-12T10:22:58.676 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2023-12-12T10:22:58.861 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-12-12T10:22:58.862 INFO:teuthology.nuke:Checking targets against current locks
2023-12-12T10:22:58.909 DEBUG:teuthology.nuke:shortname: smithi017
2023-12-12T10:22:58.909 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-12T10:22:58.929 DEBUG:teuthology.nuke:shortname: smithi028
2023-12-12T10:22:58.929 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-12-12T10:22:58.935 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488226', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-12 09:34:12.375717', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGnz2rThvQQBZ73pwGlKbdfWC5Blb2ICqPZw+FfBGNLpJT27ToA4jMDC5Vorl5ezRrqSUGi+SBzc7KHbLUDGuss='}
2023-12-12T10:22:58.949 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi028.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488226', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-12 09:34:12.377130', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMQo7phVB5Czp0vHepmn7sZcFkxu4G+Cuz31QGBumxQ81epfbK0DLF/Ac32TfpDaydhtOHs0lyUAlqkuHuP0LrM='}
2023-12-12T10:22:58.977 INFO:teuthology.orchestra.console.smithi017:Power off
2023-12-12T10:22:58.978 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-12T10:22:59.003 DEBUG:teuthology.orchestra.console.smithi017:power off output: Chassis Power Control: Down/Off
2023-12-12T10:22:59.003 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T10:22:59.016 INFO:teuthology.orchestra.console.smithi028:Power off
2023-12-12T10:22:59.016 DEBUG:teuthology.orchestra.console.smithi028:pexpect command: ipmitool -H smithi028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-12-12T10:22:59.025 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is off
2023-12-12T10:22:59.038 DEBUG:teuthology.orchestra.console.smithi028:power off output: Chassis Power Control: Down/Off
2023-12-12T10:22:59.038 DEBUG:teuthology.orchestra.console.smithi028:pexpect command: ipmitool -H smithi028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-12-12T10:22:59.059 DEBUG:teuthology.orchestra.console.smithi028:check power output: Chassis Power is off
2023-12-12T10:22:59.126 INFO:teuthology.orchestra.console.smithi017:Power off completed
2023-12-12T10:22:59.160 INFO:teuthology.orchestra.console.smithi028:Power off completed
2023-12-12T10:22:59.275 INFO:teuthology.lock.ops:unlocked: smithi017.front.sepia.ceph.com
2023-12-12T10:22:59.309 INFO:teuthology.lock.ops:unlocked: smithi028.front.sepia.ceph.com