2024-01-06T16:16:50.026 INFO:root:teuthology version: 0.0.1.dev228+g01dd171
2024-01-06T16:16:50.026 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:16:50.159 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fcf766c5280>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fcf775de940>, signals=[15])
2024-01-06T16:16:50.159 INFO:teuthology.lock.ops:Start node 'smithi062.front.sepia.ceph.com' reimaging
2024-01-06T16:16:50.159 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: reset os type and version on server
2024-01-06T16:16:50.159 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server
2024-01-06T16:16:50.185 INFO:teuthology.lock.ops:Start node 'smithi160.front.sepia.ceph.com' reimaging
2024-01-06T16:16:50.185 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: reset os type and version on server
2024-01-06T16:16:50.185 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server
2024-01-06T16:16:50.291 DEBUG:teuthology.provision.fog.smithi062:Requesting image smithi_centos_9.stream (ID 10)
2024-01-06T16:16:50.311 DEBUG:teuthology.provision.fog.smithi160:Requesting image smithi_centos_9.stream (ID 10)
2024-01-06T16:16:50.330 INFO:teuthology.provision.fog.smithi062:Scheduling deploy of centos 9.stream
2024-01-06T16:16:50.355 INFO:teuthology.provision.fog.smithi160:Scheduling deploy of centos 9.stream
2024-01-06T16:16:50.555 INFO:teuthology.orchestra.console.smithi062:Power off
2024-01-06T16:16:50.555 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-06T16:16:50.567 INFO:teuthology.orchestra.console.smithi160:Power off
2024-01-06T16:16:50.567 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-06T16:16:50.575 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off
2024-01-06T16:16:50.575 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-06T16:16:50.590 DEBUG:teuthology.orchestra.console.smithi160:power off output: Chassis Power Control: Down/Off
2024-01-06T16:16:50.591 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-06T16:16:50.599 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off
2024-01-06T16:16:50.611 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off
2024-01-06T16:16:50.700 INFO:teuthology.orchestra.console.smithi062:Power off completed
2024-01-06T16:16:50.712 INFO:teuthology.orchestra.console.smithi160:Power off completed
2024-01-06T16:16:50.801 INFO:teuthology.orchestra.console.smithi062:Power on
2024-01-06T16:16:50.801 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-06T16:16:50.813 INFO:teuthology.orchestra.console.smithi160:Power on
2024-01-06T16:16:50.814 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-01-06T16:16:50.822 DEBUG:teuthology.orchestra.console.smithi062:power on output: Chassis Power Control: Up/On
2024-01-06T16:16:50.822 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-06T16:16:50.835 DEBUG:teuthology.orchestra.console.smithi160:power on output: Chassis Power Control: Up/On
2024-01-06T16:16:50.835 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-06T16:16:50.844 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off
2024-01-06T16:16:50.859 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off
2024-01-06T16:16:54.848 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-06T16:16:54.860 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-06T16:16:54.958 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off
2024-01-06T16:16:54.968 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off
2024-01-06T16:16:58.960 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-06T16:16:58.969 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-06T16:16:59.069 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on
2024-01-06T16:16:59.078 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is on
2024-01-06T16:16:59.171 INFO:teuthology.orchestra.console.smithi062:Power on completed
2024-01-06T16:16:59.179 INFO:teuthology.orchestra.console.smithi160:Power on completed
2024-01-06T16:16:59.272 INFO:teuthology.provision.fog.smithi062:Waiting for deploy to finish
2024-01-06T16:16:59.279 INFO:teuthology.provision.fog.smithi160:Waiting for deploy to finish
2024-01-06T16:19:30.639 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-06T16:19:49.188 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62
2024-01-06T16:19:55.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-06T16:19:58.270 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62
2024-01-06T16:20:04.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-06T16:20:07.357 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62
2024-01-06T16:20:13.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-06T16:20:16.448 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62
2024-01-06T16:20:22.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-06T16:20:25.534 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62
2024-01-06T16:20:31.535 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-06T16:21:01.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-06T16:21:19.548 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160
2024-01-06T16:21:25.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-06T16:21:28.604 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160
2024-01-06T16:21:31.540 WARNING:teuthology.provision.fog.smithi062:timed out
2024-01-06T16:21:34.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-06T16:21:37.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-06T16:21:37.660 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160
2024-01-06T16:21:37.757 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-06T16:21:37.758 DEBUG:teuthology.orchestra.run.smithi062:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-06T16:21:38.084 INFO:teuthology.provision.fog.smithi062:Node is ready
2024-01-06T16:21:38.085 DEBUG:teuthology.orchestra.run.smithi062:> hostname
2024-01-06T16:21:38.100 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062.front.sepia.ceph.com
2024-01-06T16:21:38.101 DEBUG:teuthology.orchestra.run.smithi062:> grep smithi062.front.sepia.ceph.com /etc/hosts
2024-01-06T16:21:38.157 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62 smithi062.front.sepia.ceph.com smithi062
2024-01-06T16:21:38.158 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname smithi062
2024-01-06T16:21:38.273 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/smithi062.front.sepia.ceph.com/smithi062/g' /etc/hosts
2024-01-06T16:21:38.306 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/172.21.15.62/172.21.15.62/g' /etc/hosts
2024-01-06T16:21:38.376 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/smithi062.front.sepia.ceph.com/smithi062/g' /etc/hostname
2024-01-06T16:21:38.446 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname smithi062
2024-01-06T16:21:38.515 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release
2024-01-06T16:21:38.568 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream"
2024-01-06T16:21:38.568 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9"
2024-01-06T16:21:38.568 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos"
2024-01-06T16:21:38.568 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora"
2024-01-06T16:21:38.568 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9"
2024-01-06T16:21:38.568 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9"
2024-01-06T16:21:38.569 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-06T16:21:38.569 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31"
2024-01-06T16:21:38.569 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon"
2024-01-06T16:21:38.569 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-06T16:21:38.569 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/"
2024-01-06T16:21:38.569 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-06T16:21:38.569 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-06T16:21:38.569 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-06T16:21:38.570 INFO:teuthology.provision.fog.smithi062:Deploy complete!
2024-01-06T16:21:38.573 DEBUG:teuthology.parallel:result is None
2024-01-06T16:21:43.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-06T16:21:46.748 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160
2024-01-06T16:21:52.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-06T16:21:55.836 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160
2024-01-06T16:22:01.844 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-06T16:23:01.905 WARNING:teuthology.provision.fog.smithi160:timed out
2024-01-06T16:23:07.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-06T16:23:08.171 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-01-06T16:23:08.172 DEBUG:teuthology.orchestra.run.smithi160:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-01-06T16:23:08.518 INFO:teuthology.provision.fog.smithi160:Node is ready
2024-01-06T16:23:08.518 DEBUG:teuthology.orchestra.run.smithi160:> hostname
2024-01-06T16:23:08.533 INFO:teuthology.orchestra.run.smithi160.stdout:smithi160.front.sepia.ceph.com
2024-01-06T16:23:08.534 DEBUG:teuthology.orchestra.run.smithi160:> grep smithi160.front.sepia.ceph.com /etc/hosts
2024-01-06T16:23:08.589 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160 smithi160.front.sepia.ceph.com smithi160
2024-01-06T16:23:08.590 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname smithi160
2024-01-06T16:23:08.689 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/smithi160.front.sepia.ceph.com/smithi160/g' /etc/hosts
2024-01-06T16:23:08.717 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/172.21.15.160/172.21.15.160/g' /etc/hosts
2024-01-06T16:23:08.784 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/smithi160.front.sepia.ceph.com/smithi160/g' /etc/hostname
2024-01-06T16:23:08.852 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname smithi160
2024-01-06T16:23:08.919 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release
2024-01-06T16:23:08.973 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Stream"
2024-01-06T16:23:08.973 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="9"
2024-01-06T16:23:08.973 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos"
2024-01-06T16:23:08.973 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora"
2024-01-06T16:23:08.973 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="9"
2024-01-06T16:23:08.973 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el9"
2024-01-06T16:23:08.973 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-06T16:23:08.973 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31"
2024-01-06T16:23:08.973 INFO:teuthology.orchestra.run.smithi160.stdout:LOGO="fedora-logo-icon"
2024-01-06T16:23:08.974 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-06T16:23:08.974 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://centos.org/"
2024-01-06T16:23:08.974 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-06T16:23:08.974 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-06T16:23:08.974 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-06T16:23:08.974 INFO:teuthology.provision.fog.smithi160:Deploy complete!
2024-01-06T16:23:08.974 DEBUG:teuthology.parallel:result is None
2024-01-06T16:23:09.140 INFO:teuthology.lock.ops:Checking smithi062.front.sepia.ceph.com
2024-01-06T16:23:09.140 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-06T16:23:09.166 INFO:teuthology.lock.ops:Checking smithi160.front.sepia.ceph.com
2024-01-06T16:23:09.166 INFO:teuthology.lock.ops:New key found. Updating...
2024-01-06T16:23:09.183 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: set os type and version on server
2024-01-06T16:23:09.184 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com'
2024-01-06T16:23:09.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-06T16:23:09.280 DEBUG:teuthology.orchestra.run.smithi062:> true
2024-01-06T16:23:09.462 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com'
2024-01-06T16:23:09.463 DEBUG:teuthology.orchestra.run.smithi062:> uname -m
2024-01-06T16:23:09.518 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64
2024-01-06T16:23:09.518 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release
2024-01-06T16:23:09.573 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream"
2024-01-06T16:23:09.574 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9"
2024-01-06T16:23:09.574 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos"
2024-01-06T16:23:09.574 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora"
2024-01-06T16:23:09.574 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9"
2024-01-06T16:23:09.574 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9"
2024-01-06T16:23:09.574 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-06T16:23:09.574 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31"
2024-01-06T16:23:09.574 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon"
2024-01-06T16:23:09.574 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-06T16:23:09.574 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/"
2024-01-06T16:23:09.574 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-06T16:23:09.574 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-06T16:23:09.574 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-06T16:23:09.575 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server
2024-01-06T16:23:09.591 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: set os type and version on server
2024-01-06T16:23:09.591 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com'
2024-01-06T16:23:09.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-01-06T16:23:09.674 DEBUG:teuthology.orchestra.run.smithi160:> true
2024-01-06T16:23:09.743 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com'
2024-01-06T16:23:09.743 DEBUG:teuthology.orchestra.run.smithi160:> uname -m
2024-01-06T16:23:09.798 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64
2024-01-06T16:23:09.798 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release
2024-01-06T16:23:09.853 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Stream"
2024-01-06T16:23:09.853 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="9"
2024-01-06T16:23:09.853 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos"
2024-01-06T16:23:09.853 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora"
2024-01-06T16:23:09.854 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="9"
2024-01-06T16:23:09.854 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el9"
2024-01-06T16:23:09.854 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Stream 9"
2024-01-06T16:23:09.854 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31"
2024-01-06T16:23:09.854 INFO:teuthology.orchestra.run.smithi160.stdout:LOGO="fedora-logo-icon"
2024-01-06T16:23:09.854 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-01-06T16:23:09.854 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://centos.org/"
2024-01-06T16:23:09.854 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-01-06T16:23:09.854 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-01-06T16:23:09.854 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-01-06T16:23:09.854 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server
2024-01-06T16:23:09.874 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:23:09.955 INFO:teuthology.dispatcher.supervisor:Running job 7508721
2024-01-06T16:23:09.955 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/virtualenv/bin/teuthology -v --owner scheduled_rfriedma@teuthology --archive /home/teuthworker/archive/rfriedma-2024-01-06_16:00:11-rados-wip-rf-actpr-requeue-distro-default-smithi/7508721 --name rfriedma-2024-01-06_16:00:11-rados-wip-rf-actpr-requeue-distro-default-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/pggrow thrashosds-health workloads/rados_api_tests} -- /home/teuthworker/archive/rfriedma-2024-01-06_16:00:11-rados-wip-rf-actpr-requeue-distro-default-smithi/7508721/orig.config.yaml
2024-01-06T16:23:09.958 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rfriedma-2024-01-06_16:00:11-rados-wip-rf-actpr-requeue-distro-default-smithi/7508721
2024-01-06T16:23:09.959 INFO:teuthology.dispatcher.supervisor:Job PID: 3856332
2024-01-06T16:23:09.959 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-01-06T16:25:10.056 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:27:10.232 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:29:10.405 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:31:10.584 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:33:10.756 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:35:10.842 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:37:11.009 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:39:11.168 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:41:11.334 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:43:11.493 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:45:11.621 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:47:11.777 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:49:11.892 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:51:11.964 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:53:12.135 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:55:12.298 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:57:12.465 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T16:59:12.620 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:01:12.780 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:03:12.923 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:05:13.088 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:07:13.250 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:09:13.339 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:11:13.465 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:13:13.692 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:15:13.849 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:17:14.027 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:19:14.188 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:21:14.328 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:23:14.429 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:25:14.555 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:27:14.781 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:29:14.962 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:31:15.173 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:33:15.262 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:35:15.492 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:37:15.574 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:39:15.762 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:41:15.904 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:43:16.076 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:45:16.247 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:47:16.637 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:49:16.781 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:51:16.891 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:53:17.077 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:55:17.281 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:57:17.536 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T17:59:17.804 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:01:17.960 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:03:18.100 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:05:18.267 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:07:18.346 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:09:18.525 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:11:18.774 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:13:18.967 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:15:19.186 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:17:19.386 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:19:19.532 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:21:19.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:23:19.889 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:25:19.983 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:27:20.132 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:29:20.204 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:31:20.289 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:33:20.399 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:35:20.564 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:37:20.687 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:39:20.849 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:41:21.023 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:43:21.170 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:45:21.271 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:47:21.453 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:49:21.553 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:51:21.719 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:53:21.898 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:55:22.073 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:57:22.235 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T18:59:22.468 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:01:22.633 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:03:22.804 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:05:22.972 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:07:23.176 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:09:23.342 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:11:23.503 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:13:23.627 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:15:23.771 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:17:23.935 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:19:24.091 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:21:24.239 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:23:24.409 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:25:24.590 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:27:24.919 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:29:25.073 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:31:25.226 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:33:25.379 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:35:25.505 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:37:25.667 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:39:25.834 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:41:25.984 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:43:26.140 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:45:26.328 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:47:26.553 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:49:26.643 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:51:26.905 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:53:27.082 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:55:27.164 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:57:27.253 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T19:59:27.424 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T20:01:27.600 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T20:03:27.728 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T20:05:27.905 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T20:07:28.016 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-01-06T20:07:28.093 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2024-01-06T20:07:28.221 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2024-01-06T20:07:28.221 INFO:teuthology.nuke:Checking targets against current locks
2024-01-06T20:07:28.275 DEBUG:teuthology.nuke:shortname: smithi062
2024-01-06T20:07:28.275 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-06T20:07:28.287 DEBUG:teuthology.nuke:shortname: smithi160
2024-01-06T20:07:28.287 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-01-06T20:07:28.297 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rfriedma-2024-01-06_16:00:11-rados-wip-rf-actpr-requeue-distro-default-smithi/7508721', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-06 16:16:49.187174', 'locked_by': 'scheduled_rfriedma@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPFf96+vip0Nmp20Ash0+VMSC+LkZTqFE14T5IRIrFK/IMjlBBvdWjtCyzBE7WaRP85sRsdvXlzckdFLwxpY3ng='}
2024-01-06T20:07:28.307 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rfriedma-2024-01-06_16:00:11-rados-wip-rf-actpr-requeue-distro-default-smithi/7508721', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-06 16:16:49.190012', 'locked_by': 'scheduled_rfriedma@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAN+pNLTMiDjRoxeqxLqU1G0nhsWkDFPRhlBU1lHHhzVu3H4wbFv9VXKjSs7HoZ09VBthmo5AEBtHWqbXDHPtPY='}
2024-01-06T20:07:28.352 INFO:teuthology.orchestra.console.smithi160:Power off
2024-01-06T20:07:28.353 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-06T20:07:28.368 INFO:teuthology.orchestra.console.smithi062:Power off
2024-01-06T20:07:28.368 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-01-06T20:07:28.378 DEBUG:teuthology.orchestra.console.smithi160:power off output: Chassis Power Control: Down/Off
2024-01-06T20:07:28.378 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-06T20:07:28.391 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off
2024-01-06T20:07:28.392 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-01-06T20:07:28.401 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off
2024-01-06T20:07:28.413 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off
2024-01-06T20:07:28.502 INFO:teuthology.orchestra.console.smithi160:Power off completed
2024-01-06T20:07:28.515 INFO:teuthology.orchestra.console.smithi062:Power off completed
2024-01-06T20:07:28.656 INFO:teuthology.lock.ops:unlocked: smithi160.front.sepia.ceph.com
2024-01-06T20:07:28.665 INFO:teuthology.lock.ops:unlocked: smithi062.front.sepia.ceph.com