2023-11-22T08:15:23.698 INFO:root:teuthology version: 0.0.1.dev205+g2442239
2023-11-22T08:15:23.699 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:15:23.835 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f13a69d08b0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f13a74d2670>, signals=[15])
2023-11-22T08:15:23.836 INFO:teuthology.lock.ops:Start node 'smithi023.front.sepia.ceph.com' reimaging
2023-11-22T08:15:23.836 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: reset os type and version on server
2023-11-22T08:15:23.836 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server
2023-11-22T08:15:23.860 INFO:teuthology.lock.ops:Start node 'smithi137.front.sepia.ceph.com' reimaging
2023-11-22T08:15:23.860 INFO:teuthology.lock.ops:Updating [smithi137.front.sepia.ceph.com]: reset os type and version on server
2023-11-22T08:15:23.860 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server
2023-11-22T08:15:23.970 DEBUG:teuthology.provision.fog.smithi023:Requesting image smithi_centos_9.stream (ID 10)
2023-11-22T08:15:23.980 DEBUG:teuthology.provision.fog.smithi137:Requesting image smithi_centos_9.stream (ID 10)
2023-11-22T08:15:24.008 INFO:teuthology.provision.fog.smithi023:Scheduling deploy of centos 9.stream
2023-11-22T08:15:24.021 INFO:teuthology.provision.fog.smithi137:Scheduling deploy of centos 9.stream
2023-11-22T08:15:24.314 INFO:teuthology.orchestra.console.smithi023:Power off
2023-11-22T08:15:24.315 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-22T08:15:24.327 INFO:teuthology.orchestra.console.smithi137:Power off
2023-11-22T08:15:24.327 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-22T08:15:24.336 DEBUG:teuthology.orchestra.console.smithi023:power off output: Chassis Power Control: Down/Off
2023-11-22T08:15:24.336 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-22T08:15:24.347 DEBUG:teuthology.orchestra.console.smithi137:power off output: Chassis Power Control: Down/Off
2023-11-22T08:15:24.347 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-22T08:15:24.357 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on
2023-11-22T08:15:24.367 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on
2023-11-22T08:15:28.362 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-22T08:15:28.368 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-22T08:15:28.471 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on
2023-11-22T08:15:28.475 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on
2023-11-22T08:15:32.474 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-22T08:15:32.480 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-22T08:15:32.583 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on
2023-11-22T08:15:32.592 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on
2023-11-22T08:15:36.588 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-22T08:15:36.594 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-22T08:15:36.696 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off
2023-11-22T08:15:36.701 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off
2023-11-22T08:15:36.797 INFO:teuthology.orchestra.console.smithi023:Power off completed
2023-11-22T08:15:36.802 INFO:teuthology.orchestra.console.smithi137:Power off completed
2023-11-22T08:15:36.898 INFO:teuthology.orchestra.console.smithi023:Power on
2023-11-22T08:15:36.898 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-22T08:15:36.906 INFO:teuthology.orchestra.console.smithi137:Power on
2023-11-22T08:15:36.906 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-22T08:15:36.919 DEBUG:teuthology.orchestra.console.smithi023:power on output: Chassis Power Control: Up/On
2023-11-22T08:15:36.919 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-22T08:15:36.926 DEBUG:teuthology.orchestra.console.smithi137:power on output: Chassis Power Control: Up/On
2023-11-22T08:15:36.926 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-22T08:15:36.939 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off
2023-11-22T08:15:36.946 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off
2023-11-22T08:15:40.944 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-22T08:15:40.951 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-22T08:15:41.053 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off
2023-11-22T08:15:41.058 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off
2023-11-22T08:15:45.058 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-22T08:15:45.064 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-22T08:15:45.167 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on
2023-11-22T08:15:45.171 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on
2023-11-22T08:15:45.268 INFO:teuthology.orchestra.console.smithi023:Power on completed
2023-11-22T08:15:45.272 INFO:teuthology.orchestra.console.smithi137:Power on completed
2023-11-22T08:15:45.369 INFO:teuthology.provision.fog.smithi023:Waiting for deploy to finish
2023-11-22T08:15:45.373 INFO:teuthology.provision.fog.smithi137:Waiting for deploy to finish
2023-11-22T08:19:49.959 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:19:53.026 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-11-22T08:19:59.031 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:20:02.082 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-11-22T08:20:08.088 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:20:11.138 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-11-22T08:20:17.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:20:20.194 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-11-22T08:20:26.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:20:29.282 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-11-22T08:20:35.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:20:35.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:20:38.338 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-11-22T08:20:38.434 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137
2023-11-22T08:20:44.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:20:44.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:20:47.394 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-11-22T08:20:47.490 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137
2023-11-22T08:20:53.402 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:20:53.491 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:20:56.483 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-11-22T08:20:56.546 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137
2023-11-22T08:21:02.487 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:21:02.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:21:05.538 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23
2023-11-22T08:21:05.602 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137
2023-11-22T08:21:11.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:21:11.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:21:14.658 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137
2023-11-22T08:21:20.662 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:21:23.714 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137
2023-11-22T08:21:29.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:21:48.258 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137
2023-11-22T08:21:54.265 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:21:57.351 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137
2023-11-22T08:22:03.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:22:06.434 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137
2023-11-22T08:22:11.553 WARNING:teuthology.provision.fog.smithi023:timed out
2023-11-22T08:22:12.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:22:12.945 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-22T08:22:12.945 DEBUG:teuthology.orchestra.run.smithi137:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-22T08:22:17.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:22:17.781 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-22T08:22:17.781 DEBUG:teuthology.orchestra.run.smithi023:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-22T08:22:18.116 INFO:teuthology.provision.fog.smithi023:Node is ready
2023-11-22T08:22:18.116 DEBUG:teuthology.orchestra.run.smithi023:> hostname
2023-11-22T08:22:18.131 INFO:teuthology.orchestra.run.smithi023.stdout:smithi023.front.sepia.ceph.com
2023-11-22T08:22:18.132 DEBUG:teuthology.orchestra.run.smithi023:> grep smithi023.front.sepia.ceph.com /etc/hosts
2023-11-22T08:22:18.187 INFO:teuthology.orchestra.run.smithi023.stdout:172.21.15.23 smithi023.front.sepia.ceph.com smithi023
2023-11-22T08:22:18.188 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023
2023-11-22T08:22:18.299 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hosts
2023-11-22T08:22:18.331 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/172.21.15.23/172.21.15.23/g' /etc/hosts
2023-11-22T08:22:18.402 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hostname
2023-11-22T08:22:18.473 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023
2023-11-22T08:22:18.543 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release
2023-11-22T08:22:18.597 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Stream"
2023-11-22T08:22:18.598 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="9"
2023-11-22T08:22:18.598 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos"
2023-11-22T08:22:18.598 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora"
2023-11-22T08:22:18.598 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="9"
2023-11-22T08:22:18.598 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el9"
2023-11-22T08:22:18.598 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Stream 9"
2023-11-22T08:22:18.598 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31"
2023-11-22T08:22:18.598 INFO:teuthology.orchestra.run.smithi023.stdout:LOGO="fedora-logo-icon"
2023-11-22T08:22:18.598 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-11-22T08:22:18.599 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://centos.org/"
2023-11-22T08:22:18.599 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-22T08:22:18.599 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-11-22T08:22:18.599 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-11-22T08:22:18.599 INFO:teuthology.provision.fog.smithi023:Deploy complete!
2023-11-22T08:22:18.600 DEBUG:teuthology.parallel:result is None
2023-11-22T08:22:28.378 INFO:teuthology.provision.fog.smithi137:Node is ready
2023-11-22T08:22:28.379 DEBUG:teuthology.orchestra.run.smithi137:> hostname
2023-11-22T08:22:28.436 INFO:teuthology.orchestra.run.smithi137.stdout:smithi137.front.sepia.ceph.com
2023-11-22T08:22:28.436 DEBUG:teuthology.orchestra.run.smithi137:> grep smithi137.front.sepia.ceph.com /etc/hosts
2023-11-22T08:22:28.494 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137 smithi137.front.sepia.ceph.com smithi137
2023-11-22T08:22:28.494 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname smithi137
2023-11-22T08:22:28.598 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i -e 's/smithi137.front.sepia.ceph.com/smithi137/g' /etc/hosts
2023-11-22T08:22:28.630 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i -e 's/172.21.15.137/172.21.15.137/g' /etc/hosts
2023-11-22T08:22:28.699 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i -e 's/smithi137.front.sepia.ceph.com/smithi137/g' /etc/hostname
2023-11-22T08:22:28.768 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname smithi137
2023-11-22T08:22:28.836 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release
2023-11-22T08:22:28.891 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream"
2023-11-22T08:22:28.891 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9"
2023-11-22T08:22:28.891 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos"
2023-11-22T08:22:28.891 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora"
2023-11-22T08:22:28.891 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9"
2023-11-22T08:22:28.891 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9"
2023-11-22T08:22:28.891 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9"
2023-11-22T08:22:28.891 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31"
2023-11-22T08:22:28.891 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon"
2023-11-22T08:22:28.892 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-11-22T08:22:28.892 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/"
2023-11-22T08:22:28.892 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-22T08:22:28.892 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-11-22T08:22:28.892 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-11-22T08:22:28.892 INFO:teuthology.provision.fog.smithi137:Deploy complete!
2023-11-22T08:22:28.892 DEBUG:teuthology.parallel:result is None
2023-11-22T08:22:29.062 INFO:teuthology.lock.ops:Checking smithi023.front.sepia.ceph.com
2023-11-22T08:22:29.062 INFO:teuthology.lock.ops:New key found. Updating...
2023-11-22T08:22:29.085 INFO:teuthology.lock.ops:Checking smithi137.front.sepia.ceph.com
2023-11-22T08:22:29.085 INFO:teuthology.lock.ops:New key found. Updating...
2023-11-22T08:22:29.111 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: set os type and version on server
2023-11-22T08:22:29.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com'
2023-11-22T08:22:29.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:22:29.207 DEBUG:teuthology.orchestra.run.smithi023:> true
2023-11-22T08:22:29.388 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi023.front.sepia.ceph.com'
2023-11-22T08:22:29.389 DEBUG:teuthology.orchestra.run.smithi023:> uname -m
2023-11-22T08:22:29.444 INFO:teuthology.orchestra.run.smithi023.stdout:x86_64
2023-11-22T08:22:29.444 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release
2023-11-22T08:22:29.499 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Stream"
2023-11-22T08:22:29.499 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="9"
2023-11-22T08:22:29.499 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos"
2023-11-22T08:22:29.499 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora"
2023-11-22T08:22:29.499 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="9"
2023-11-22T08:22:29.499 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el9"
2023-11-22T08:22:29.499 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Stream 9"
2023-11-22T08:22:29.499 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31"
2023-11-22T08:22:29.499 INFO:teuthology.orchestra.run.smithi023.stdout:LOGO="fedora-logo-icon"
2023-11-22T08:22:29.499 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-11-22T08:22:29.499 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://centos.org/"
2023-11-22T08:22:29.499 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-22T08:22:29.500 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-11-22T08:22:29.500 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-11-22T08:22:29.500 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server
2023-11-22T08:22:29.535 INFO:teuthology.lock.ops:Updating [smithi137.front.sepia.ceph.com]: set os type and version on server
2023-11-22T08:22:29.536 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com'
2023-11-22T08:22:29.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-22T08:22:29.617 DEBUG:teuthology.orchestra.run.smithi137:> true
2023-11-22T08:22:29.686 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com'
2023-11-22T08:22:29.686 DEBUG:teuthology.orchestra.run.smithi137:> uname -m
2023-11-22T08:22:29.743 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64
2023-11-22T08:22:29.743 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release
2023-11-22T08:22:29.799 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream"
2023-11-22T08:22:29.799 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9"
2023-11-22T08:22:29.799 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos"
2023-11-22T08:22:29.799 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora"
2023-11-22T08:22:29.799 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9"
2023-11-22T08:22:29.799 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9"
2023-11-22T08:22:29.800 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9"
2023-11-22T08:22:29.800 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31"
2023-11-22T08:22:29.800 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon"
2023-11-22T08:22:29.800 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-11-22T08:22:29.800 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/"
2023-11-22T08:22:29.800 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-11-22T08:22:29.800 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-11-22T08:22:29.800 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-11-22T08:22:29.800 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server
2023-11-22T08:22:29.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:22:29.895 INFO:teuthology.dispatcher.supervisor:Running job 7464306
2023-11-22T08:22:29.895 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/virtualenv/bin/teuthology -v --owner scheduled_matan@teuthology --archive /home/teuthworker/archive/matan-2023-11-22_08:14:32-rados:verify-main-distro-default-smithi/7464306 --name matan-2023-11-22_08:14:32-rados:verify-main-distro-default-smithi --description rados:verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/mon_recovery validater/valgrind} -- /home/teuthworker/archive/matan-2023-11-22_08:14:32-rados:verify-main-distro-default-smithi/7464306/orig.config.yaml
2023-11-22T08:22:29.897 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/matan-2023-11-22_08:14:32-rados:verify-main-distro-default-smithi/7464306
2023-11-22T08:22:29.898 INFO:teuthology.dispatcher.supervisor:Job PID: 3340615
2023-11-22T08:22:29.898 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-11-22T08:24:29.998 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:26:30.173 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:28:30.356 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:30:30.524 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:32:30.707 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:34:30.887 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:36:31.084 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:38:31.162 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:40:31.298 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:42:31.474 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:44:31.646 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:46:31.788 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:48:31.919 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:50:32.068 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:52:32.210 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:54:32.371 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:56:32.474 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T08:58:32.652 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:00:32.804 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:02:32.954 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:04:33.105 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:06:33.223 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:08:33.369 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:10:33.536 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:12:33.678 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:14:33.833 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:16:33.991 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:18:34.148 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:20:34.296 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:22:34.410 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:24:34.563 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:26:34.727 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:28:34.891 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:30:35.059 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:32:35.231 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:34:35.406 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:36:35.561 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:38:35.731 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:40:35.794 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:42:35.882 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:44:36.045 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:46:36.210 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:48:36.371 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:50:36.543 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:52:36.719 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:54:36.839 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:56:36.989 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T09:58:37.150 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:00:37.311 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:02:37.479 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:04:37.627 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:06:37.799 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:08:37.968 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:10:38.179 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:12:39.898 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:14:40.040 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:16:42.299 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:18:42.450 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:20:42.612 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:22:42.776 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:24:42.932 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:26:43.094 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:28:43.249 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:30:43.407 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:32:43.559 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:34:43.727 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:36:43.893 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:38:44.041 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:40:44.198 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:42:44.357 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:44:44.526 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:46:44.687 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:48:44.830 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:50:44.995 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-22T10:50:45.060 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2023-11-22T10:50:45.919 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-11-22T10:50:45.919 INFO:teuthology.nuke:Checking targets against current locks
2023-11-22T10:50:45.965 DEBUG:teuthology.nuke:shortname: smithi023
2023-11-22T10:50:45.966 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-22T10:50:45.990 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi023.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-11-22_08:14:32-rados:verify-main-distro-default-smithi/7464306', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-22 08:15:22.967272', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH1cjqITMUwdoTwKrv4xsp/GpbJq8K9XdxegYlPf5Thm7N1HKYvJLjYnAAbIS1NWN/pyPVv/o8L2PMcV1VaSMx4='}
2023-11-22T10:50:45.993 DEBUG:teuthology.nuke:shortname: smithi137
2023-11-22T10:50:45.993 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-22T10:50:46.015 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2023-11-22_08:14:32-rados:verify-main-distro-default-smithi/7464306', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-22 08:15:22.966205', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG7JLZcQeuknui8T6sO4th53aVvungldwh3WOGVdmKx9tio/HEIgwFkX22mmSRI8PB+QI6IhjQ7uuwL1gd42fJw='}
2023-11-22T10:50:46.046 INFO:teuthology.orchestra.console.smithi023:Power off
2023-11-22T10:50:46.046 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-22T10:50:46.067 DEBUG:teuthology.orchestra.console.smithi023:power off output: Chassis Power Control: Down/Off
2023-11-22T10:50:46.067 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-22T10:50:46.079 INFO:teuthology.orchestra.console.smithi137:Power off
2023-11-22T10:50:46.079 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-22T10:50:46.088 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off
2023-11-22T10:50:46.099 DEBUG:teuthology.orchestra.console.smithi137:power off output: Chassis Power Control: Down/Off
2023-11-22T10:50:46.099 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-22T10:50:46.119 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off
2023-11-22T10:50:46.189 INFO:teuthology.orchestra.console.smithi023:Power off completed
2023-11-22T10:50:46.220 INFO:teuthology.orchestra.console.smithi137:Power off completed
2023-11-22T10:50:46.340 INFO:teuthology.lock.ops:unlocked: smithi023.front.sepia.ceph.com
2023-11-22T10:50:46.365 INFO:teuthology.lock.ops:unlocked: smithi137.front.sepia.ceph.com