2026-02-24T13:54:18.883 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-24T13:54:18.883 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T13:54:18.959 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T13:54:19.216 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x79bed982e150>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x79bed9069300>, signals=[15])
2026-02-24T13:54:19.216 INFO:teuthology.lock.ops:Start node 'trial058.front.sepia.ceph.com' reimaging
2026-02-24T13:54:19.216 INFO:teuthology.lock.ops:Updating [trial058.front.sepia.ceph.com]: reset os type and version on server
2026-02-24T13:54:19.216 INFO:teuthology.lock.ops:Updating trial058.front.sepia.ceph.com on lock server
2026-02-24T13:54:19.274 INFO:teuthology.lock.ops:Start node 'trial097.front.sepia.ceph.com' reimaging
2026-02-24T13:54:19.275 INFO:teuthology.lock.ops:Updating [trial097.front.sepia.ceph.com]: reset os type and version on server
2026-02-24T13:54:19.275 INFO:teuthology.lock.ops:Updating trial097.front.sepia.ceph.com on lock server
2026-02-24T13:54:19.334 INFO:teuthology.lock.ops:Start node 'trial191.front.sepia.ceph.com' reimaging
2026-02-24T13:54:19.334 INFO:teuthology.lock.ops:Updating [trial191.front.sepia.ceph.com]: reset os type and version on server
2026-02-24T13:54:19.334 INFO:teuthology.lock.ops:Updating trial191.front.sepia.ceph.com on lock server
2026-02-24T13:54:19.534 DEBUG:teuthology.provision.fog.trial058:Requesting image trial_centos_9.stream (ID 3)
2026-02-24T13:54:19.572 DEBUG:teuthology.provision.fog.trial097:Requesting image trial_centos_9.stream (ID 3)
2026-02-24T13:54:19.586 INFO:teuthology.provision.fog.trial058:Scheduling deploy of centos 9.stream
2026-02-24T13:54:19.589 DEBUG:teuthology.provision.fog.trial191:Requesting image trial_centos_9.stream (ID 3)
2026-02-24T13:54:19.623 INFO:teuthology.provision.fog.trial097:Scheduling deploy of centos 9.stream
2026-02-24T13:54:19.641 INFO:teuthology.provision.fog.trial191:Scheduling deploy of centos 9.stream
2026-02-24T13:54:21.043 INFO:teuthology.orchestra.console.trial058:Power off
2026-02-24T13:54:21.043 DEBUG:teuthology.orchestra.console.trial058:pexpect command: ipmitool -H trial058.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T13:54:21.056 INFO:teuthology.orchestra.console.trial191:Power off
2026-02-24T13:54:21.056 DEBUG:teuthology.orchestra.console.trial191:pexpect command: ipmitool -H trial191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T13:54:21.067 INFO:teuthology.orchestra.console.trial097:Power off
2026-02-24T13:54:21.067 DEBUG:teuthology.orchestra.console.trial097:pexpect command: ipmitool -H trial097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T13:54:21.081 DEBUG:teuthology.orchestra.console.trial058:power off output: Chassis Power Control: Down/Off
2026-02-24T13:54:21.095 DEBUG:teuthology.orchestra.console.trial191:power off output: Chassis Power Control: Down/Off
2026-02-24T13:54:21.106 DEBUG:teuthology.orchestra.console.trial097:power off output: Chassis Power Control: Down/Off
2026-02-24T13:54:25.086 DEBUG:teuthology.orchestra.console.trial058:pexpect command: ipmitool -H trial058.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T13:54:25.095 DEBUG:teuthology.orchestra.console.trial191:pexpect command: ipmitool -H trial191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T13:54:25.107 DEBUG:teuthology.orchestra.console.trial097:pexpect command: ipmitool -H trial097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T13:54:25.196 DEBUG:teuthology.orchestra.console.trial058:check power output: Chassis Power is off
2026-02-24T13:54:25.197 INFO:teuthology.orchestra.console.trial058:Power off completed
2026-02-24T13:54:25.205 DEBUG:teuthology.orchestra.console.trial191:check power output: Chassis Power is off
2026-02-24T13:54:25.205 INFO:teuthology.orchestra.console.trial191:Power off completed
2026-02-24T13:54:25.214 DEBUG:teuthology.orchestra.console.trial097:check power output: Chassis Power is off
2026-02-24T13:54:25.215 INFO:teuthology.orchestra.console.trial097:Power off completed
2026-02-24T13:54:25.298 INFO:teuthology.orchestra.console.trial058:Power on
2026-02-24T13:54:25.298 DEBUG:teuthology.orchestra.console.trial058:pexpect command: ipmitool -H trial058.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-24T13:54:25.308 INFO:teuthology.orchestra.console.trial191:Power on
2026-02-24T13:54:25.308 DEBUG:teuthology.orchestra.console.trial191:pexpect command: ipmitool -H trial191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-24T13:54:25.316 INFO:teuthology.orchestra.console.trial097:Power on
2026-02-24T13:54:25.317 DEBUG:teuthology.orchestra.console.trial097:pexpect command: ipmitool -H trial097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-24T13:54:25.340 DEBUG:teuthology.orchestra.console.trial058:power on output: Chassis Power Control: Up/On
2026-02-24T13:54:25.348 DEBUG:teuthology.orchestra.console.trial191:power on output: Chassis Power Control: Up/On
2026-02-24T13:54:25.356 DEBUG:teuthology.orchestra.console.trial097:power on output: Chassis Power Control: Up/On
2026-02-24T13:54:29.344 DEBUG:teuthology.orchestra.console.trial058:pexpect command: ipmitool -H trial058.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T13:54:29.350 DEBUG:teuthology.orchestra.console.trial191:pexpect command: ipmitool -H trial191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T13:54:29.357 DEBUG:teuthology.orchestra.console.trial097:pexpect command: ipmitool -H trial097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T13:54:29.452 DEBUG:teuthology.orchestra.console.trial058:check power output: Chassis Power is off
2026-02-24T13:54:29.458 DEBUG:teuthology.orchestra.console.trial191:check power output: Chassis Power is on
2026-02-24T13:54:29.458 INFO:teuthology.orchestra.console.trial191:Power on completed
2026-02-24T13:54:29.464 DEBUG:teuthology.orchestra.console.trial097:check power output: Chassis Power is on
2026-02-24T13:54:29.464 INFO:teuthology.orchestra.console.trial097:Power on completed
2026-02-24T13:54:29.559 INFO:teuthology.provision.fog.trial191:Waiting for deploy to finish
2026-02-24T13:54:29.564 INFO:teuthology.provision.fog.trial097:Waiting for deploy to finish
2026-02-24T13:54:33.456 DEBUG:teuthology.orchestra.console.trial058:pexpect command: ipmitool -H trial058.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T13:54:33.567 DEBUG:teuthology.orchestra.console.trial058:check power output: Chassis Power is on
2026-02-24T13:54:33.567 INFO:teuthology.orchestra.console.trial058:Power on completed
2026-02-24T13:54:33.668 INFO:teuthology.provision.fog.trial058:Waiting for deploy to finish
2026-02-24T13:56:17.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T13:56:21.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial058.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T13:56:24.064 WARNING:teuthology.provision.fog.trial058:[Errno None] Unable to connect to port 22 on 10.20.193.58
2026-02-24T13:56:30.069 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial058.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T13:56:32.749 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T13:56:33.151 WARNING:teuthology.provision.fog.trial058:[Errno None] Unable to connect to port 22 on 10.20.193.58
2026-02-24T13:56:35.841 WARNING:teuthology.provision.fog.trial097:[Errno None] Unable to connect to port 22 on 10.20.193.97
2026-02-24T13:56:39.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial058.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T13:56:41.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T13:56:42.240 WARNING:teuthology.provision.fog.trial058:[Errno None] Unable to connect to port 22 on 10.20.193.58
2026-02-24T13:56:44.928 WARNING:teuthology.provision.fog.trial097:[Errno None] Unable to connect to port 22 on 10.20.193.97
2026-02-24T13:56:48.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial058.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T13:56:50.932 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T13:56:51.327 WARNING:teuthology.provision.fog.trial058:[Errno None] Unable to connect to port 22 on 10.20.193.58
2026-02-24T13:56:54.016 WARNING:teuthology.provision.fog.trial097:[Errno None] Unable to connect to port 22 on 10.20.193.97
2026-02-24T13:56:57.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial058.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T13:57:00.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T13:57:00.415 WARNING:teuthology.provision.fog.trial058:[Errno None] Unable to connect to port 22 on 10.20.193.58
2026-02-24T13:57:03.103 WARNING:teuthology.provision.fog.trial097:[Errno None] Unable to connect to port 22 on 10.20.193.97
2026-02-24T13:57:06.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial058.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T13:57:09.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T13:57:17.465 WARNING:teuthology.provision.fog.trial191:timed out
2026-02-24T13:57:23.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T13:57:30.760 ERROR:teuthology.orchestra.connection:Error authenticating with trial191.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-24T13:57:34.860 ERROR:teuthology.orchestra.connection:Error authenticating with trial191.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-24T13:57:41.944 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-24T13:57:41.944 DEBUG:teuthology.orchestra.run.trial191:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-24T13:57:42.097 INFO:teuthology.provision.fog.trial191:Node is ready
2026-02-24T13:57:42.097 DEBUG:teuthology.orchestra.run.trial191:> hostname
2026-02-24T13:57:42.152 INFO:teuthology.orchestra.run.trial191.stdout:trial191.front.sepia.ceph.com
2026-02-24T13:57:42.153 DEBUG:teuthology.orchestra.run.trial191:> grep trial191.front.sepia.ceph.com /etc/hosts
2026-02-24T13:57:42.209 INFO:teuthology.orchestra.run.trial191.stdout:10.20.193.191 trial191.front.sepia.ceph.com trial191
2026-02-24T13:57:42.210 DEBUG:teuthology.orchestra.run.trial191:> sudo hostname trial191
2026-02-24T13:57:42.278 DEBUG:teuthology.orchestra.run.trial191:> sudo sed -i -e 's/trial191.front.sepia.ceph.com/trial191/g' /etc/hosts
2026-02-24T13:57:42.343 DEBUG:teuthology.orchestra.run.trial191:> sudo sed -i -e 's/10.20.193.191/10.20.193.191/g' /etc/hosts
2026-02-24T13:57:42.408 DEBUG:teuthology.orchestra.run.trial191:> sudo sed -i -e 's/trial191.front.sepia.ceph.com/trial191/g' /etc/hostname
2026-02-24T13:57:42.523 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-24T13:57:42.523 DEBUG:teuthology.orchestra.run.trial058:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-24T13:57:42.524 DEBUG:teuthology.orchestra.run.trial191:> sudo hostname trial191
2026-02-24T13:57:42.587 DEBUG:teuthology.orchestra.run.trial191:> cat /etc/os-release
2026-02-24T13:57:42.643 INFO:teuthology.orchestra.run.trial191.stdout:NAME="CentOS Stream"
2026-02-24T13:57:42.643 INFO:teuthology.orchestra.run.trial191.stdout:VERSION="9"
2026-02-24T13:57:42.643 INFO:teuthology.orchestra.run.trial191.stdout:ID="centos"
2026-02-24T13:57:42.643 INFO:teuthology.orchestra.run.trial191.stdout:ID_LIKE="rhel fedora"
2026-02-24T13:57:42.643 INFO:teuthology.orchestra.run.trial191.stdout:VERSION_ID="9"
2026-02-24T13:57:42.643 INFO:teuthology.orchestra.run.trial191.stdout:PLATFORM_ID="platform:el9"
2026-02-24T13:57:42.643 INFO:teuthology.orchestra.run.trial191.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T13:57:42.643 INFO:teuthology.orchestra.run.trial191.stdout:ANSI_COLOR="0;31"
2026-02-24T13:57:42.643 INFO:teuthology.orchestra.run.trial191.stdout:LOGO="fedora-logo-icon"
2026-02-24T13:57:42.643 INFO:teuthology.orchestra.run.trial191.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T13:57:42.643 INFO:teuthology.orchestra.run.trial191.stdout:HOME_URL="https://centos.org/"
2026-02-24T13:57:42.643 INFO:teuthology.orchestra.run.trial191.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T13:57:42.643 INFO:teuthology.orchestra.run.trial191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T13:57:42.643 INFO:teuthology.orchestra.run.trial191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T13:57:42.643 INFO:teuthology.provision.fog.trial191:Deploy complete!
2026-02-24T13:57:42.644 DEBUG:teuthology.parallel:result is None
2026-02-24T13:57:42.687 INFO:teuthology.provision.fog.trial058:Node is ready
2026-02-24T13:57:42.687 DEBUG:teuthology.orchestra.run.trial058:> hostname
2026-02-24T13:57:42.741 INFO:teuthology.orchestra.run.trial058.stdout:trial058.front.sepia.ceph.com
2026-02-24T13:57:42.742 DEBUG:teuthology.orchestra.run.trial058:> grep trial058.front.sepia.ceph.com /etc/hosts
2026-02-24T13:57:42.798 INFO:teuthology.orchestra.run.trial058.stdout:10.20.193.58 trial058.front.sepia.ceph.com trial058
2026-02-24T13:57:42.798 DEBUG:teuthology.orchestra.run.trial058:> sudo hostname trial058
2026-02-24T13:57:42.866 DEBUG:teuthology.orchestra.run.trial058:> sudo sed -i -e 's/trial058.front.sepia.ceph.com/trial058/g' /etc/hosts
2026-02-24T13:57:42.931 DEBUG:teuthology.orchestra.run.trial058:> sudo sed -i -e 's/10.20.193.58/10.20.193.58/g' /etc/hosts
2026-02-24T13:57:42.994 DEBUG:teuthology.orchestra.run.trial058:> sudo sed -i -e 's/trial058.front.sepia.ceph.com/trial058/g' /etc/hostname
2026-02-24T13:57:43.058 DEBUG:teuthology.orchestra.run.trial058:> sudo hostname trial058
2026-02-24T13:57:43.121 DEBUG:teuthology.orchestra.run.trial058:> cat /etc/os-release
2026-02-24T13:57:43.177 INFO:teuthology.orchestra.run.trial058.stdout:NAME="CentOS Stream"
2026-02-24T13:57:43.178 INFO:teuthology.orchestra.run.trial058.stdout:VERSION="9"
2026-02-24T13:57:43.178 INFO:teuthology.orchestra.run.trial058.stdout:ID="centos"
2026-02-24T13:57:43.178 INFO:teuthology.orchestra.run.trial058.stdout:ID_LIKE="rhel fedora"
2026-02-24T13:57:43.178 INFO:teuthology.orchestra.run.trial058.stdout:VERSION_ID="9"
2026-02-24T13:57:43.178 INFO:teuthology.orchestra.run.trial058.stdout:PLATFORM_ID="platform:el9"
2026-02-24T13:57:43.178 INFO:teuthology.orchestra.run.trial058.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T13:57:43.178 INFO:teuthology.orchestra.run.trial058.stdout:ANSI_COLOR="0;31"
2026-02-24T13:57:43.178 INFO:teuthology.orchestra.run.trial058.stdout:LOGO="fedora-logo-icon"
2026-02-24T13:57:43.179 INFO:teuthology.orchestra.run.trial058.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T13:57:43.179 INFO:teuthology.orchestra.run.trial058.stdout:HOME_URL="https://centos.org/"
2026-02-24T13:57:43.179 INFO:teuthology.orchestra.run.trial058.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T13:57:43.179 INFO:teuthology.orchestra.run.trial058.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T13:57:43.179 INFO:teuthology.orchestra.run.trial058.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T13:57:43.179 INFO:teuthology.provision.fog.trial058:Deploy complete!
2026-02-24T13:57:43.180 DEBUG:teuthology.parallel:result is None
2026-02-24T13:57:45.037 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-24T13:57:45.037 DEBUG:teuthology.orchestra.run.trial097:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-24T13:57:45.197 INFO:teuthology.provision.fog.trial097:Node is ready
2026-02-24T13:57:45.198 DEBUG:teuthology.orchestra.run.trial097:> hostname
2026-02-24T13:57:45.252 INFO:teuthology.orchestra.run.trial097.stdout:trial097.front.sepia.ceph.com
2026-02-24T13:57:45.253 DEBUG:teuthology.orchestra.run.trial097:> grep trial097.front.sepia.ceph.com /etc/hosts
2026-02-24T13:57:45.310 INFO:teuthology.orchestra.run.trial097.stdout:10.20.193.97 trial097.front.sepia.ceph.com trial097
2026-02-24T13:57:45.310 DEBUG:teuthology.orchestra.run.trial097:> sudo hostname trial097
2026-02-24T13:57:45.380 DEBUG:teuthology.orchestra.run.trial097:> sudo sed -i -e 's/trial097.front.sepia.ceph.com/trial097/g' /etc/hosts
2026-02-24T13:57:45.444 DEBUG:teuthology.orchestra.run.trial097:> sudo sed -i -e 's/10.20.193.97/10.20.193.97/g' /etc/hosts
2026-02-24T13:57:45.510 DEBUG:teuthology.orchestra.run.trial097:> sudo sed -i -e 's/trial097.front.sepia.ceph.com/trial097/g' /etc/hostname
2026-02-24T13:57:45.574 DEBUG:teuthology.orchestra.run.trial097:> sudo hostname trial097
2026-02-24T13:57:45.637 DEBUG:teuthology.orchestra.run.trial097:> cat /etc/os-release
2026-02-24T13:57:45.694 INFO:teuthology.orchestra.run.trial097.stdout:NAME="CentOS Stream"
2026-02-24T13:57:45.695 INFO:teuthology.orchestra.run.trial097.stdout:VERSION="9"
2026-02-24T13:57:45.695 INFO:teuthology.orchestra.run.trial097.stdout:ID="centos"
2026-02-24T13:57:45.695 INFO:teuthology.orchestra.run.trial097.stdout:ID_LIKE="rhel fedora"
2026-02-24T13:57:45.695 INFO:teuthology.orchestra.run.trial097.stdout:VERSION_ID="9"
2026-02-24T13:57:45.695 INFO:teuthology.orchestra.run.trial097.stdout:PLATFORM_ID="platform:el9"
2026-02-24T13:57:45.695 INFO:teuthology.orchestra.run.trial097.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T13:57:45.695 INFO:teuthology.orchestra.run.trial097.stdout:ANSI_COLOR="0;31"
2026-02-24T13:57:45.695 INFO:teuthology.orchestra.run.trial097.stdout:LOGO="fedora-logo-icon"
2026-02-24T13:57:45.695 INFO:teuthology.orchestra.run.trial097.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T13:57:45.696 INFO:teuthology.orchestra.run.trial097.stdout:HOME_URL="https://centos.org/"
2026-02-24T13:57:45.696 INFO:teuthology.orchestra.run.trial097.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T13:57:45.696 INFO:teuthology.orchestra.run.trial097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T13:57:45.696 INFO:teuthology.orchestra.run.trial097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T13:57:45.696 INFO:teuthology.provision.fog.trial097:Deploy complete!
2026-02-24T13:57:45.697 DEBUG:teuthology.parallel:result is None
2026-02-24T13:57:46.222 INFO:teuthology.lock.ops:Checking trial058.front.sepia.ceph.com
2026-02-24T13:57:46.222 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-24T13:57:46.283 INFO:teuthology.lock.ops:Checking trial097.front.sepia.ceph.com
2026-02-24T13:57:46.283 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-24T13:57:46.340 INFO:teuthology.lock.ops:Checking trial191.front.sepia.ceph.com
2026-02-24T13:57:46.341 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-24T13:57:46.398 INFO:teuthology.lock.ops:Updating [trial058.front.sepia.ceph.com]: set os type and version on server
2026-02-24T13:57:46.398 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial058.front.sepia.ceph.com'
2026-02-24T13:57:46.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial058.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T13:57:46.461 DEBUG:teuthology.orchestra.run.trial058:> true
2026-02-24T13:57:46.539 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial058.front.sepia.ceph.com'
2026-02-24T13:57:46.539 DEBUG:teuthology.orchestra.run.trial058:> uname -m
2026-02-24T13:57:46.594 INFO:teuthology.orchestra.run.trial058.stdout:x86_64
2026-02-24T13:57:46.594 DEBUG:teuthology.orchestra.run.trial058:> cat /etc/os-release
2026-02-24T13:57:46.650 INFO:teuthology.orchestra.run.trial058.stdout:NAME="CentOS Stream"
2026-02-24T13:57:46.650 INFO:teuthology.orchestra.run.trial058.stdout:VERSION="9"
2026-02-24T13:57:46.650 INFO:teuthology.orchestra.run.trial058.stdout:ID="centos"
2026-02-24T13:57:46.650 INFO:teuthology.orchestra.run.trial058.stdout:ID_LIKE="rhel fedora"
2026-02-24T13:57:46.651 INFO:teuthology.orchestra.run.trial058.stdout:VERSION_ID="9"
2026-02-24T13:57:46.651 INFO:teuthology.orchestra.run.trial058.stdout:PLATFORM_ID="platform:el9"
2026-02-24T13:57:46.651 INFO:teuthology.orchestra.run.trial058.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T13:57:46.651 INFO:teuthology.orchestra.run.trial058.stdout:ANSI_COLOR="0;31"
2026-02-24T13:57:46.651 INFO:teuthology.orchestra.run.trial058.stdout:LOGO="fedora-logo-icon"
2026-02-24T13:57:46.651 INFO:teuthology.orchestra.run.trial058.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T13:57:46.651 INFO:teuthology.orchestra.run.trial058.stdout:HOME_URL="https://centos.org/"
2026-02-24T13:57:46.651 INFO:teuthology.orchestra.run.trial058.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T13:57:46.652 INFO:teuthology.orchestra.run.trial058.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T13:57:46.652 INFO:teuthology.orchestra.run.trial058.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T13:57:46.653 INFO:teuthology.lock.ops:Updating trial058.front.sepia.ceph.com on lock server
2026-02-24T13:57:46.747 INFO:teuthology.lock.ops:Updating [trial097.front.sepia.ceph.com]: set os type and version on server
2026-02-24T13:57:46.747 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial097.front.sepia.ceph.com'
2026-02-24T13:57:46.748 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T13:57:46.809 DEBUG:teuthology.orchestra.run.trial097:> true
2026-02-24T13:57:46.879 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial097.front.sepia.ceph.com'
2026-02-24T13:57:46.879 DEBUG:teuthology.orchestra.run.trial097:> uname -m
2026-02-24T13:57:46.935 INFO:teuthology.orchestra.run.trial097.stdout:x86_64
2026-02-24T13:57:46.935 DEBUG:teuthology.orchestra.run.trial097:> cat /etc/os-release
2026-02-24T13:57:46.992 INFO:teuthology.orchestra.run.trial097.stdout:NAME="CentOS Stream"
2026-02-24T13:57:46.992 INFO:teuthology.orchestra.run.trial097.stdout:VERSION="9"
2026-02-24T13:57:46.992 INFO:teuthology.orchestra.run.trial097.stdout:ID="centos"
2026-02-24T13:57:46.992 INFO:teuthology.orchestra.run.trial097.stdout:ID_LIKE="rhel fedora"
2026-02-24T13:57:46.992 INFO:teuthology.orchestra.run.trial097.stdout:VERSION_ID="9"
2026-02-24T13:57:46.992 INFO:teuthology.orchestra.run.trial097.stdout:PLATFORM_ID="platform:el9"
2026-02-24T13:57:46.992 INFO:teuthology.orchestra.run.trial097.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T13:57:46.993 INFO:teuthology.orchestra.run.trial097.stdout:ANSI_COLOR="0;31"
2026-02-24T13:57:46.993 INFO:teuthology.orchestra.run.trial097.stdout:LOGO="fedora-logo-icon"
2026-02-24T13:57:46.993 INFO:teuthology.orchestra.run.trial097.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T13:57:46.993 INFO:teuthology.orchestra.run.trial097.stdout:HOME_URL="https://centos.org/"
2026-02-24T13:57:46.993 INFO:teuthology.orchestra.run.trial097.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T13:57:46.993 INFO:teuthology.orchestra.run.trial097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T13:57:46.993 INFO:teuthology.orchestra.run.trial097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T13:57:46.994 INFO:teuthology.lock.ops:Updating trial097.front.sepia.ceph.com on lock server
2026-02-24T13:57:47.111 INFO:teuthology.lock.ops:Updating [trial191.front.sepia.ceph.com]: set os type and version on server
2026-02-24T13:57:47.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial191.front.sepia.ceph.com'
2026-02-24T13:57:47.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T13:57:47.183 DEBUG:teuthology.orchestra.run.trial191:> true
2026-02-24T13:57:47.256 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial191.front.sepia.ceph.com'
2026-02-24T13:57:47.256 DEBUG:teuthology.orchestra.run.trial191:> uname -m
2026-02-24T13:57:47.311 INFO:teuthology.orchestra.run.trial191.stdout:x86_64
2026-02-24T13:57:47.312 DEBUG:teuthology.orchestra.run.trial191:> cat /etc/os-release
2026-02-24T13:57:47.367 INFO:teuthology.orchestra.run.trial191.stdout:NAME="CentOS Stream"
2026-02-24T13:57:47.368 INFO:teuthology.orchestra.run.trial191.stdout:VERSION="9"
2026-02-24T13:57:47.368 INFO:teuthology.orchestra.run.trial191.stdout:ID="centos"
2026-02-24T13:57:47.368 INFO:teuthology.orchestra.run.trial191.stdout:ID_LIKE="rhel fedora"
2026-02-24T13:57:47.368 INFO:teuthology.orchestra.run.trial191.stdout:VERSION_ID="9"
2026-02-24T13:57:47.368 INFO:teuthology.orchestra.run.trial191.stdout:PLATFORM_ID="platform:el9"
2026-02-24T13:57:47.368 INFO:teuthology.orchestra.run.trial191.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T13:57:47.368 INFO:teuthology.orchestra.run.trial191.stdout:ANSI_COLOR="0;31"
2026-02-24T13:57:47.368 INFO:teuthology.orchestra.run.trial191.stdout:LOGO="fedora-logo-icon"
2026-02-24T13:57:47.368 INFO:teuthology.orchestra.run.trial191.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T13:57:47.368 INFO:teuthology.orchestra.run.trial191.stdout:HOME_URL="https://centos.org/"
2026-02-24T13:57:47.368 INFO:teuthology.orchestra.run.trial191.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T13:57:47.369 INFO:teuthology.orchestra.run.trial191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T13:57:47.369 INFO:teuthology.orchestra.run.trial191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T13:57:47.369 INFO:teuthology.lock.ops:Updating trial191.front.sepia.ceph.com on lock server
2026-02-24T13:57:47.462 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T13:57:47.552 INFO:teuthology.dispatcher.supervisor:Running job 67860
2026-02-24T13:57:47.552 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/bin/teuthology -v --owner scheduled_cjames@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/cjames-2026-02-24_10:01:51-rados-main-distro-default-trial/67860 --name cjames-2026-02-24_10:01:51-rados-main-distro-default-trial --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds} ec_direct_reads/ec_direct_reads_on ec_optimizations/ec_optimizations_on mon_election/connectivity msgr-failures/fastclose objectstore/{bluestore/{alloc$/{stupid} base mem$/{normal-1} onode-segment$/{none} write$/{v1/{compr$/{no$/{no}} v1}}}} rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/mnt/teuthology/cjames-2026-02-24_10:01:51-rados-main-distro-default-trial/67860/orig.config.yaml
2026-02-24T13:57:47.553 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/cjames-2026-02-24_10:01:51-rados-main-distro-default-trial/67860
2026-02-24T13:57:47.553 INFO:teuthology.dispatcher.supervisor:Job PID: 822616
2026-02-24T13:57:47.553 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-24T13:59:47.618 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T14:01:47.795 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T14:03:47.975 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T14:05:48.155 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T14:07:48.352 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T14:09:48.525 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T14:11:48.709 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T14:13:48.889 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T14:15:49.037 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T14:17:49.217 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T14:19:49.398 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T14:21:49.604 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T14:23:49.774 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T14:23:49.864 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-24T14:23:50.038 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-24T14:23:50.686 INFO:teuthology.orchestra.console.trial097:Power off
2026-02-24T14:23:50.686 DEBUG:teuthology.orchestra.console.trial097:pexpect command: ipmitool -H trial097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T14:23:50.741 DEBUG:teuthology.orchestra.console.trial097:power off output: Chassis Power Control: Down/Off
2026-02-24T14:23:50.752 INFO:teuthology.orchestra.console.trial191:Power off
2026-02-24T14:23:50.752 DEBUG:teuthology.orchestra.console.trial191:pexpect command: ipmitool -H trial191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T14:23:50.809 DEBUG:teuthology.orchestra.console.trial191:power off output: Chassis Power Control: Down/Off
2026-02-24T14:23:50.821 INFO:teuthology.orchestra.console.trial058:Power off
2026-02-24T14:23:50.822 DEBUG:teuthology.orchestra.console.trial058:pexpect command: ipmitool -H trial058.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T14:23:50.859 DEBUG:teuthology.orchestra.console.trial058:power off output: Chassis Power Control: Down/Off
2026-02-24T14:23:54.745 DEBUG:teuthology.orchestra.console.trial097:pexpect command: ipmitool -H trial097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T14:23:54.810 DEBUG:teuthology.orchestra.console.trial191:pexpect command: ipmitool -H trial191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T14:23:54.854 DEBUG:teuthology.orchestra.console.trial097:check power output: Chassis Power is on
2026-02-24T14:23:54.860 DEBUG:teuthology.orchestra.console.trial058:pexpect command: ipmitool -H trial058.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T14:23:54.921 DEBUG:teuthology.orchestra.console.trial191:check power output: Chassis Power is on
2026-02-24T14:23:54.971 DEBUG:teuthology.orchestra.console.trial058:check power output: Chassis Power is on
2026-02-24T14:23:58.859 DEBUG:teuthology.orchestra.console.trial097:pexpect command: ipmitool -H trial097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T14:23:58.922 DEBUG:teuthology.orchestra.console.trial191:pexpect command: ipmitool -H trial191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T14:23:58.968 DEBUG:teuthology.orchestra.console.trial097:check power output: Chassis Power is off
2026-02-24T14:23:58.969 INFO:teuthology.orchestra.console.trial097:Power off completed
2026-02-24T14:23:58.972 DEBUG:teuthology.orchestra.console.trial058:pexpect command: ipmitool -H trial058.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T14:23:59.033 DEBUG:teuthology.orchestra.console.trial191:check power output: Chassis Power is off
2026-02-24T14:23:59.033 INFO:teuthology.orchestra.console.trial191:Power off completed
2026-02-24T14:23:59.168 DEBUG:teuthology.orchestra.console.trial058:check power output: Chassis Power is off
2026-02-24T14:23:59.168 INFO:teuthology.orchestra.console.trial058:Power off completed
2026-02-24T14:23:59.227 INFO:teuthology.lock.ops:unlocked: trial191.front.sepia.ceph.com
2026-02-24T14:23:59.339 INFO:teuthology.lock.ops:unlocked: trial058.front.sepia.ceph.com
2026-02-24T14:24:00.613 INFO:teuthology.lock.ops:unlocked: trial097.front.sepia.ceph.com