2026-02-20T13:54:59.515 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-20T13:54:59.515 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-20T13:54:59.589 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-20T13:54:59.846 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x761643b46e70>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x761643386ca0>, signals=[15])
2026-02-20T13:54:59.846 INFO:teuthology.lock.ops:Start node 'trial104.front.sepia.ceph.com' reimaging
2026-02-20T13:54:59.846 INFO:teuthology.lock.ops:Updating [trial104.front.sepia.ceph.com]: reset os type and version on server
2026-02-20T13:54:59.846 INFO:teuthology.lock.ops:Updating trial104.front.sepia.ceph.com on lock server
2026-02-20T13:54:59.905 INFO:teuthology.lock.ops:Start node 'trial125.front.sepia.ceph.com' reimaging
2026-02-20T13:54:59.906 INFO:teuthology.lock.ops:Updating [trial125.front.sepia.ceph.com]: reset os type and version on server
2026-02-20T13:54:59.906 INFO:teuthology.lock.ops:Updating trial125.front.sepia.ceph.com on lock server
2026-02-20T13:54:59.964 INFO:teuthology.lock.ops:Start node 'trial145.front.sepia.ceph.com' reimaging
2026-02-20T13:54:59.964 INFO:teuthology.lock.ops:Updating [trial145.front.sepia.ceph.com]: reset os type and version on server
2026-02-20T13:54:59.964 INFO:teuthology.lock.ops:Updating trial145.front.sepia.ceph.com on lock server
2026-02-20T13:55:00.192 DEBUG:teuthology.provision.fog.trial125:Requesting image trial_centos_9.stream (ID 3)
2026-02-20T13:55:00.192 DEBUG:teuthology.provision.fog.trial104:Requesting image trial_centos_9.stream (ID 3)
2026-02-20T13:55:00.241 INFO:teuthology.provision.fog.trial125:Scheduling deploy of centos 9.stream
2026-02-20T13:55:00.246 INFO:teuthology.provision.fog.trial104:Scheduling deploy of centos 9.stream
2026-02-20T13:55:00.252 DEBUG:teuthology.provision.fog.trial145:Requesting image trial_centos_9.stream (ID 3)
2026-02-20T13:55:00.303 INFO:teuthology.provision.fog.trial145:Scheduling deploy of centos 9.stream
2026-02-20T13:55:01.551 INFO:teuthology.orchestra.console.trial104:Power off
2026-02-20T13:55:01.551 DEBUG:teuthology.orchestra.console.trial104:pexpect command: ipmitool -H trial104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-20T13:55:01.563 INFO:teuthology.orchestra.console.trial145:Power off
2026-02-20T13:55:01.564 DEBUG:teuthology.orchestra.console.trial145:pexpect command: ipmitool -H trial145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-20T13:55:01.573 INFO:teuthology.orchestra.console.trial125:Power off
2026-02-20T13:55:01.573 DEBUG:teuthology.orchestra.console.trial125:pexpect command: ipmitool -H trial125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-20T13:55:01.592 DEBUG:teuthology.orchestra.console.trial104:power off output: Chassis Power Control: Down/Off
2026-02-20T13:55:01.600 DEBUG:teuthology.orchestra.console.trial145:power off output: Chassis Power Control: Down/Off
2026-02-20T13:55:01.610 DEBUG:teuthology.orchestra.console.trial125:power off output: Chassis Power Control: Down/Off
2026-02-20T13:55:05.597 DEBUG:teuthology.orchestra.console.trial104:pexpect command: ipmitool -H trial104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-20T13:55:05.602 DEBUG:teuthology.orchestra.console.trial145:pexpect command: ipmitool -H trial145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-20T13:55:05.611 DEBUG:teuthology.orchestra.console.trial125:pexpect command: ipmitool -H trial125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-20T13:55:05.704 DEBUG:teuthology.orchestra.console.trial104:check power output: Chassis Power is off
2026-02-20T13:55:05.705 INFO:teuthology.orchestra.console.trial104:Power off completed
2026-02-20T13:55:05.709 DEBUG:teuthology.orchestra.console.trial145:check power output: Chassis Power is off
2026-02-20T13:55:05.709 INFO:teuthology.orchestra.console.trial145:Power off completed
2026-02-20T13:55:05.718 DEBUG:teuthology.orchestra.console.trial125:check power output: Chassis Power is off
2026-02-20T13:55:05.718 INFO:teuthology.orchestra.console.trial125:Power off completed
2026-02-20T13:55:05.806 INFO:teuthology.orchestra.console.trial104:Power on
2026-02-20T13:55:05.806 DEBUG:teuthology.orchestra.console.trial104:pexpect command: ipmitool -H trial104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-20T13:55:05.814 INFO:teuthology.orchestra.console.trial145:Power on
2026-02-20T13:55:05.815 DEBUG:teuthology.orchestra.console.trial145:pexpect command: ipmitool -H trial145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-20T13:55:05.822 INFO:teuthology.orchestra.console.trial125:Power on
2026-02-20T13:55:05.822 DEBUG:teuthology.orchestra.console.trial125:pexpect command: ipmitool -H trial125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-20T13:55:05.845 DEBUG:teuthology.orchestra.console.trial104:power on output: Chassis Power Control: Up/On
2026-02-20T13:55:05.853 DEBUG:teuthology.orchestra.console.trial145:power on output: Chassis Power Control: Up/On
2026-02-20T13:55:05.861 DEBUG:teuthology.orchestra.console.trial125:power on output: Chassis Power Control: Up/On
2026-02-20T13:55:09.850 DEBUG:teuthology.orchestra.console.trial104:pexpect command: ipmitool -H trial104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-20T13:55:09.857 DEBUG:teuthology.orchestra.console.trial145:pexpect command: ipmitool -H trial145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-20T13:55:09.862 DEBUG:teuthology.orchestra.console.trial125:pexpect command: ipmitool -H trial125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-20T13:55:09.958 DEBUG:teuthology.orchestra.console.trial104:check power output: Chassis Power is on
2026-02-20T13:55:09.959 INFO:teuthology.orchestra.console.trial104:Power on completed
2026-02-20T13:55:09.964 DEBUG:teuthology.orchestra.console.trial145:check power output: Chassis Power is on
2026-02-20T13:55:09.964 INFO:teuthology.orchestra.console.trial145:Power on completed
2026-02-20T13:55:09.968 DEBUG:teuthology.orchestra.console.trial125:check power output: Chassis Power is on
2026-02-20T13:55:09.969 INFO:teuthology.orchestra.console.trial125:Power on completed
2026-02-20T13:55:10.060 INFO:teuthology.provision.fog.trial104:Waiting for deploy to finish
2026-02-20T13:55:10.065 INFO:teuthology.provision.fog.trial145:Waiting for deploy to finish
2026-02-20T13:55:10.070 INFO:teuthology.provision.fog.trial125:Waiting for deploy to finish
2026-02-20T13:56:57.844 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-20T13:56:58.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-20T13:56:58.004 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-20T13:56:58.032 ERROR:teuthology.orchestra.connection:Error authenticating with trial145.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2026-02-20T13:56:58.130 ERROR:teuthology.orchestra.connection:Error authenticating with trial125.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2026-02-20T13:57:58.005 WARNING:teuthology.provision.fog.trial104:timed out
2026-02-20T13:58:02.034 WARNING:teuthology.provision.fog.trial145:timed out
2026-02-20T13:58:02.132 WARNING:teuthology.provision.fog.trial125:timed out
2026-02-20T13:58:04.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-20T13:58:08.036 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-20T13:58:08.134 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-20T13:58:09.284 ERROR:teuthology.orchestra.connection:Error authenticating with trial104.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-20T13:58:13.410 ERROR:teuthology.orchestra.connection:Error authenticating with trial104.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-20T13:58:15.397 ERROR:teuthology.orchestra.connection:Error authenticating with trial125.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-20T13:58:15.535 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-20T13:58:15.535 DEBUG:teuthology.orchestra.run.trial145:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-20T13:58:19.507 ERROR:teuthology.orchestra.connection:Error authenticating with trial125.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-20T13:58:20.528 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-20T13:58:20.528 DEBUG:teuthology.orchestra.run.trial104:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-20T13:58:20.688 INFO:teuthology.provision.fog.trial104:Node is ready
2026-02-20T13:58:20.688 DEBUG:teuthology.orchestra.run.trial104:> hostname
2026-02-20T13:58:20.745 INFO:teuthology.orchestra.run.trial104.stdout:trial104.front.sepia.ceph.com
2026-02-20T13:58:20.746 DEBUG:teuthology.orchestra.run.trial104:> grep trial104.front.sepia.ceph.com /etc/hosts
2026-02-20T13:58:20.804 INFO:teuthology.orchestra.run.trial104.stdout:10.20.193.104 trial104.front.sepia.ceph.com trial104
2026-02-20T13:58:20.804 DEBUG:teuthology.orchestra.run.trial104:> sudo hostname trial104
2026-02-20T13:58:20.877 DEBUG:teuthology.orchestra.run.trial104:> sudo sed -i -e 's/trial104.front.sepia.ceph.com/trial104/g' /etc/hosts
2026-02-20T13:58:20.942 DEBUG:teuthology.orchestra.run.trial104:> sudo sed -i -e 's/10.20.193.104/10.20.193.104/g' /etc/hosts
2026-02-20T13:58:21.008 DEBUG:teuthology.orchestra.run.trial104:> sudo sed -i -e 's/trial104.front.sepia.ceph.com/trial104/g' /etc/hostname
2026-02-20T13:58:21.075 DEBUG:teuthology.orchestra.run.trial104:> sudo hostname trial104
2026-02-20T13:58:21.146 DEBUG:teuthology.orchestra.run.trial104:> cat /etc/os-release
2026-02-20T13:58:21.209 INFO:teuthology.orchestra.run.trial104.stdout:NAME="CentOS Stream"
2026-02-20T13:58:21.209 INFO:teuthology.orchestra.run.trial104.stdout:VERSION="9"
2026-02-20T13:58:21.209 INFO:teuthology.orchestra.run.trial104.stdout:ID="centos"
2026-02-20T13:58:21.209 INFO:teuthology.orchestra.run.trial104.stdout:ID_LIKE="rhel fedora"
2026-02-20T13:58:21.210 INFO:teuthology.orchestra.run.trial104.stdout:VERSION_ID="9"
2026-02-20T13:58:21.210 INFO:teuthology.orchestra.run.trial104.stdout:PLATFORM_ID="platform:el9"
2026-02-20T13:58:21.210 INFO:teuthology.orchestra.run.trial104.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-20T13:58:21.210 INFO:teuthology.orchestra.run.trial104.stdout:ANSI_COLOR="0;31"
2026-02-20T13:58:21.210 INFO:teuthology.orchestra.run.trial104.stdout:LOGO="fedora-logo-icon"
2026-02-20T13:58:21.210 INFO:teuthology.orchestra.run.trial104.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-20T13:58:21.210 INFO:teuthology.orchestra.run.trial104.stdout:HOME_URL="https://centos.org/"
2026-02-20T13:58:21.210 INFO:teuthology.orchestra.run.trial104.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-20T13:58:21.210 INFO:teuthology.orchestra.run.trial104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-20T13:58:21.210 INFO:teuthology.orchestra.run.trial104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-20T13:58:21.211 INFO:teuthology.provision.fog.trial104:Deploy complete!
2026-02-20T13:58:21.211 DEBUG:teuthology.parallel:result is None
2026-02-20T13:58:25.616 INFO:teuthology.provision.fog.trial145:Node is ready
2026-02-20T13:58:25.617 DEBUG:teuthology.orchestra.run.trial145:> hostname
2026-02-20T13:58:25.634 INFO:teuthology.orchestra.run.trial145.stdout:trial145.front.sepia.ceph.com
2026-02-20T13:58:25.635 DEBUG:teuthology.orchestra.run.trial145:> grep trial145.front.sepia.ceph.com /etc/hosts
2026-02-20T13:58:25.691 INFO:teuthology.orchestra.run.trial145.stdout:10.20.193.145 trial145.front.sepia.ceph.com trial145
2026-02-20T13:58:25.692 DEBUG:teuthology.orchestra.run.trial145:> sudo hostname trial145
2026-02-20T13:58:25.844 DEBUG:teuthology.orchestra.run.trial145:> sudo sed -i -e 's/trial145.front.sepia.ceph.com/trial145/g' /etc/hosts
2026-02-20T13:58:25.888 DEBUG:teuthology.orchestra.run.trial145:> sudo sed -i -e 's/10.20.193.145/10.20.193.145/g' /etc/hosts
2026-02-20T13:58:25.954 DEBUG:teuthology.orchestra.run.trial145:> sudo sed -i -e 's/trial145.front.sepia.ceph.com/trial145/g' /etc/hostname
2026-02-20T13:58:26.021 DEBUG:teuthology.orchestra.run.trial145:> sudo hostname trial145
2026-02-20T13:58:26.086 DEBUG:teuthology.orchestra.run.trial145:> cat /etc/os-release
2026-02-20T13:58:26.144 INFO:teuthology.orchestra.run.trial145.stdout:NAME="CentOS Stream"
2026-02-20T13:58:26.144 INFO:teuthology.orchestra.run.trial145.stdout:VERSION="9"
2026-02-20T13:58:26.144 INFO:teuthology.orchestra.run.trial145.stdout:ID="centos"
2026-02-20T13:58:26.144 INFO:teuthology.orchestra.run.trial145.stdout:ID_LIKE="rhel fedora"
2026-02-20T13:58:26.145 INFO:teuthology.orchestra.run.trial145.stdout:VERSION_ID="9"
2026-02-20T13:58:26.145 INFO:teuthology.orchestra.run.trial145.stdout:PLATFORM_ID="platform:el9"
2026-02-20T13:58:26.145 INFO:teuthology.orchestra.run.trial145.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-20T13:58:26.145 INFO:teuthology.orchestra.run.trial145.stdout:ANSI_COLOR="0;31"
2026-02-20T13:58:26.145 INFO:teuthology.orchestra.run.trial145.stdout:LOGO="fedora-logo-icon"
2026-02-20T13:58:26.145 INFO:teuthology.orchestra.run.trial145.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-20T13:58:26.145 INFO:teuthology.orchestra.run.trial145.stdout:HOME_URL="https://centos.org/"
2026-02-20T13:58:26.145 INFO:teuthology.orchestra.run.trial145.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-20T13:58:26.145 INFO:teuthology.orchestra.run.trial145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-20T13:58:26.145 INFO:teuthology.orchestra.run.trial145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-20T13:58:26.146 INFO:teuthology.provision.fog.trial145:Deploy complete!
2026-02-20T13:58:26.147 DEBUG:teuthology.parallel:result is None
2026-02-20T13:58:26.616 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-20T13:58:26.616 DEBUG:teuthology.orchestra.run.trial125:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-20T13:58:26.776 INFO:teuthology.provision.fog.trial125:Node is ready
2026-02-20T13:58:26.776 DEBUG:teuthology.orchestra.run.trial125:> hostname
2026-02-20T13:58:26.831 INFO:teuthology.orchestra.run.trial125.stdout:trial125.front.sepia.ceph.com
2026-02-20T13:58:26.832 DEBUG:teuthology.orchestra.run.trial125:> grep trial125.front.sepia.ceph.com /etc/hosts
2026-02-20T13:58:26.888 INFO:teuthology.orchestra.run.trial125.stdout:10.20.193.125 trial125.front.sepia.ceph.com trial125
2026-02-20T13:58:26.889 DEBUG:teuthology.orchestra.run.trial125:> sudo hostname trial125
2026-02-20T13:58:26.958 DEBUG:teuthology.orchestra.run.trial125:> sudo sed -i -e 's/trial125.front.sepia.ceph.com/trial125/g' /etc/hosts
2026-02-20T13:58:27.023 DEBUG:teuthology.orchestra.run.trial125:> sudo sed -i -e 's/10.20.193.125/10.20.193.125/g' /etc/hosts
2026-02-20T13:58:27.087 DEBUG:teuthology.orchestra.run.trial125:> sudo sed -i -e 's/trial125.front.sepia.ceph.com/trial125/g' /etc/hostname
2026-02-20T13:58:27.152 DEBUG:teuthology.orchestra.run.trial125:> sudo hostname trial125
2026-02-20T13:58:27.216 DEBUG:teuthology.orchestra.run.trial125:> cat /etc/os-release
2026-02-20T13:58:27.273 INFO:teuthology.orchestra.run.trial125.stdout:NAME="CentOS Stream"
2026-02-20T13:58:27.273 INFO:teuthology.orchestra.run.trial125.stdout:VERSION="9"
2026-02-20T13:58:27.273 INFO:teuthology.orchestra.run.trial125.stdout:ID="centos"
2026-02-20T13:58:27.273 INFO:teuthology.orchestra.run.trial125.stdout:ID_LIKE="rhel fedora"
2026-02-20T13:58:27.273 INFO:teuthology.orchestra.run.trial125.stdout:VERSION_ID="9"
2026-02-20T13:58:27.274 INFO:teuthology.orchestra.run.trial125.stdout:PLATFORM_ID="platform:el9"
2026-02-20T13:58:27.274 INFO:teuthology.orchestra.run.trial125.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-20T13:58:27.274 INFO:teuthology.orchestra.run.trial125.stdout:ANSI_COLOR="0;31"
2026-02-20T13:58:27.274 INFO:teuthology.orchestra.run.trial125.stdout:LOGO="fedora-logo-icon"
2026-02-20T13:58:27.274 INFO:teuthology.orchestra.run.trial125.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-20T13:58:27.274 INFO:teuthology.orchestra.run.trial125.stdout:HOME_URL="https://centos.org/"
2026-02-20T13:58:27.274 INFO:teuthology.orchestra.run.trial125.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-20T13:58:27.274 INFO:teuthology.orchestra.run.trial125.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-20T13:58:27.274 INFO:teuthology.orchestra.run.trial125.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-20T13:58:27.275 INFO:teuthology.provision.fog.trial125:Deploy complete!
2026-02-20T13:58:27.276 DEBUG:teuthology.parallel:result is None
2026-02-20T13:58:27.806 INFO:teuthology.lock.ops:Checking trial104.front.sepia.ceph.com
2026-02-20T13:58:27.807 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-20T13:58:27.871 INFO:teuthology.lock.ops:Checking trial125.front.sepia.ceph.com
2026-02-20T13:58:27.871 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-20T13:58:27.929 INFO:teuthology.lock.ops:Checking trial145.front.sepia.ceph.com
2026-02-20T13:58:27.929 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-20T13:58:27.985 INFO:teuthology.lock.ops:Updating [trial104.front.sepia.ceph.com]: set os type and version on server
2026-02-20T13:58:27.985 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial104.front.sepia.ceph.com'
2026-02-20T13:58:27.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-20T13:58:28.044 DEBUG:teuthology.orchestra.run.trial104:> true
2026-02-20T13:58:28.120 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial104.front.sepia.ceph.com'
2026-02-20T13:58:28.120 DEBUG:teuthology.orchestra.run.trial104:> uname -m
2026-02-20T13:58:28.175 INFO:teuthology.orchestra.run.trial104.stdout:x86_64
2026-02-20T13:58:28.176 DEBUG:teuthology.orchestra.run.trial104:> cat /etc/os-release
2026-02-20T13:58:28.231 INFO:teuthology.orchestra.run.trial104.stdout:NAME="CentOS Stream"
2026-02-20T13:58:28.232 INFO:teuthology.orchestra.run.trial104.stdout:VERSION="9"
2026-02-20T13:58:28.232 INFO:teuthology.orchestra.run.trial104.stdout:ID="centos"
2026-02-20T13:58:28.232 INFO:teuthology.orchestra.run.trial104.stdout:ID_LIKE="rhel fedora"
2026-02-20T13:58:28.232 INFO:teuthology.orchestra.run.trial104.stdout:VERSION_ID="9"
2026-02-20T13:58:28.232 INFO:teuthology.orchestra.run.trial104.stdout:PLATFORM_ID="platform:el9"
2026-02-20T13:58:28.232 INFO:teuthology.orchestra.run.trial104.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-20T13:58:28.232 INFO:teuthology.orchestra.run.trial104.stdout:ANSI_COLOR="0;31"
2026-02-20T13:58:28.232 INFO:teuthology.orchestra.run.trial104.stdout:LOGO="fedora-logo-icon"
2026-02-20T13:58:28.232 INFO:teuthology.orchestra.run.trial104.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-20T13:58:28.233 INFO:teuthology.orchestra.run.trial104.stdout:HOME_URL="https://centos.org/"
2026-02-20T13:58:28.233 INFO:teuthology.orchestra.run.trial104.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-20T13:58:28.233 INFO:teuthology.orchestra.run.trial104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-20T13:58:28.233 INFO:teuthology.orchestra.run.trial104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-20T13:58:28.234 INFO:teuthology.lock.ops:Updating trial104.front.sepia.ceph.com on lock server
2026-02-20T13:58:28.335 INFO:teuthology.lock.ops:Updating [trial125.front.sepia.ceph.com]: set os type and version on server
2026-02-20T13:58:28.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial125.front.sepia.ceph.com'
2026-02-20T13:58:28.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-20T13:58:28.405 DEBUG:teuthology.orchestra.run.trial125:> true
2026-02-20T13:58:28.486 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial125.front.sepia.ceph.com'
2026-02-20T13:58:28.486 DEBUG:teuthology.orchestra.run.trial125:> uname -m
2026-02-20T13:58:28.542 INFO:teuthology.orchestra.run.trial125.stdout:x86_64
2026-02-20T13:58:28.543 DEBUG:teuthology.orchestra.run.trial125:> cat /etc/os-release
2026-02-20T13:58:28.598 INFO:teuthology.orchestra.run.trial125.stdout:NAME="CentOS Stream"
2026-02-20T13:58:28.598 INFO:teuthology.orchestra.run.trial125.stdout:VERSION="9"
2026-02-20T13:58:28.599 INFO:teuthology.orchestra.run.trial125.stdout:ID="centos"
2026-02-20T13:58:28.599 INFO:teuthology.orchestra.run.trial125.stdout:ID_LIKE="rhel fedora"
2026-02-20T13:58:28.599 INFO:teuthology.orchestra.run.trial125.stdout:VERSION_ID="9"
2026-02-20T13:58:28.599 INFO:teuthology.orchestra.run.trial125.stdout:PLATFORM_ID="platform:el9"
2026-02-20T13:58:28.599 INFO:teuthology.orchestra.run.trial125.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-20T13:58:28.599 INFO:teuthology.orchestra.run.trial125.stdout:ANSI_COLOR="0;31"
2026-02-20T13:58:28.599 INFO:teuthology.orchestra.run.trial125.stdout:LOGO="fedora-logo-icon"
2026-02-20T13:58:28.599 INFO:teuthology.orchestra.run.trial125.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-20T13:58:28.599 INFO:teuthology.orchestra.run.trial125.stdout:HOME_URL="https://centos.org/"
2026-02-20T13:58:28.599 INFO:teuthology.orchestra.run.trial125.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-20T13:58:28.599 INFO:teuthology.orchestra.run.trial125.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-20T13:58:28.600 INFO:teuthology.orchestra.run.trial125.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-20T13:58:28.600 INFO:teuthology.lock.ops:Updating trial125.front.sepia.ceph.com on lock server
2026-02-20T13:58:28.704 INFO:teuthology.lock.ops:Updating [trial145.front.sepia.ceph.com]: set os type and version on server
2026-02-20T13:58:28.704 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial145.front.sepia.ceph.com'
2026-02-20T13:58:28.705 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-20T13:58:28.775 DEBUG:teuthology.orchestra.run.trial145:> true
2026-02-20T13:58:28.845 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial145.front.sepia.ceph.com'
2026-02-20T13:58:28.846 DEBUG:teuthology.orchestra.run.trial145:> uname -m
2026-02-20T13:58:28.901 INFO:teuthology.orchestra.run.trial145.stdout:x86_64
2026-02-20T13:58:28.901 DEBUG:teuthology.orchestra.run.trial145:> cat /etc/os-release
2026-02-20T13:58:28.957 INFO:teuthology.orchestra.run.trial145.stdout:NAME="CentOS Stream"
2026-02-20T13:58:28.957 INFO:teuthology.orchestra.run.trial145.stdout:VERSION="9"
2026-02-20T13:58:28.958 INFO:teuthology.orchestra.run.trial145.stdout:ID="centos"
2026-02-20T13:58:28.958 INFO:teuthology.orchestra.run.trial145.stdout:ID_LIKE="rhel fedora"
2026-02-20T13:58:28.958 INFO:teuthology.orchestra.run.trial145.stdout:VERSION_ID="9"
2026-02-20T13:58:28.958 INFO:teuthology.orchestra.run.trial145.stdout:PLATFORM_ID="platform:el9"
2026-02-20T13:58:28.958 INFO:teuthology.orchestra.run.trial145.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-20T13:58:28.958 INFO:teuthology.orchestra.run.trial145.stdout:ANSI_COLOR="0;31"
2026-02-20T13:58:28.958 INFO:teuthology.orchestra.run.trial145.stdout:LOGO="fedora-logo-icon"
2026-02-20T13:58:28.958 INFO:teuthology.orchestra.run.trial145.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-20T13:58:28.958 INFO:teuthology.orchestra.run.trial145.stdout:HOME_URL="https://centos.org/"
2026-02-20T13:58:28.958 INFO:teuthology.orchestra.run.trial145.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-20T13:58:28.958 INFO:teuthology.orchestra.run.trial145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-20T13:58:28.959 INFO:teuthology.orchestra.run.trial145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-20T13:58:28.959 INFO:teuthology.lock.ops:Updating trial145.front.sepia.ceph.com on lock server
2026-02-20T13:58:29.065 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-20T13:58:29.160 INFO:teuthology.dispatcher.supervisor:Running job 60316
2026-02-20T13:58:29.160 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/bin/teuthology -v --owner scheduled_vshankar@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/vshankar-2026-02-20_09:38:02-fs-wip-vshankar-testing-20260219.125903-testing-default-trial/60316 --name vshankar-2026-02-20_09:38:02-fs-wip-vshankar-testing-20260219.125903-testing-default-trial --description fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/squid 1-client 2-upgrade 3-compat_client/no}} -- /home/teuthworker/mnt/teuthology/vshankar-2026-02-20_09:38:02-fs-wip-vshankar-testing-20260219.125903-testing-default-trial/60316/orig.config.yaml
2026-02-20T13:58:29.161 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/vshankar-2026-02-20_09:38:02-fs-wip-vshankar-testing-20260219.125903-testing-default-trial/60316
2026-02-20T13:58:29.161 INFO:teuthology.dispatcher.supervisor:Job PID: 2298858
2026-02-20T13:58:29.161 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-20T14:00:29.162 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-20T14:02:29.285 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-20T14:04:29.400 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-20T14:06:29.519 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-20T14:08:29.649 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-20T14:10:29.770 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-20T14:12:29.899 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-20T14:14:29.982 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-20T14:16:30.102 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-20T14:18:30.229 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-20T14:18:30.349 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-20T14:18:32.806 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-20T14:18:37.791 INFO:teuthology.orchestra.console.trial125:Power off
2026-02-20T14:18:37.791 DEBUG:teuthology.orchestra.console.trial125:pexpect command: ipmitool -H trial125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-20T14:18:37.833 DEBUG:teuthology.orchestra.console.trial125:power off output: Chassis Power Control: Down/Off
2026-02-20T14:18:40.727 INFO:teuthology.orchestra.console.trial104:Power off
2026-02-20T14:18:40.727 DEBUG:teuthology.orchestra.console.trial104:pexpect command: ipmitool -H trial104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-20T14:18:40.765 DEBUG:teuthology.orchestra.console.trial104:power off output: Chassis Power Control: Down/Off
2026-02-20T14:18:41.835 DEBUG:teuthology.orchestra.console.trial125:pexpect command: ipmitool -H trial125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-20T14:18:41.946 DEBUG:teuthology.orchestra.console.trial125:check power output: Chassis Power is on
2026-02-20T14:18:44.766 DEBUG:teuthology.orchestra.console.trial104:pexpect command: ipmitool -H trial104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-20T14:18:44.877 DEBUG:teuthology.orchestra.console.trial104:check power output: Chassis Power is on
2026-02-20T14:18:45.948 DEBUG:teuthology.orchestra.console.trial125:pexpect command: ipmitool -H trial125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-20T14:18:46.059 DEBUG:teuthology.orchestra.console.trial125:check power output: Chassis Power is off
2026-02-20T14:18:46.060 INFO:teuthology.orchestra.console.trial125:Power off completed
2026-02-20T14:18:46.270 INFO:teuthology.lock.ops:unlocked: trial125.front.sepia.ceph.com
2026-02-20T14:18:48.878 DEBUG:teuthology.orchestra.console.trial104:pexpect command: ipmitool -H trial104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-20T14:18:48.987 DEBUG:teuthology.orchestra.console.trial104:check power output: Chassis Power is off
2026-02-20T14:18:48.987 INFO:teuthology.orchestra.console.trial104:Power off completed
2026-02-20T14:18:49.219 INFO:teuthology.lock.ops:unlocked: trial104.front.sepia.ceph.com
2026-02-20T14:18:50.396 INFO:teuthology.orchestra.console.trial145:Power off
2026-02-20T14:18:50.396 DEBUG:teuthology.orchestra.console.trial145:pexpect command: ipmitool -H trial145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-20T14:18:50.439 DEBUG:teuthology.orchestra.console.trial145:power off output: Chassis Power Control: Down/Off
2026-02-20T14:18:54.440 DEBUG:teuthology.orchestra.console.trial145:pexpect command: ipmitool -H trial145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-20T14:18:54.551 DEBUG:teuthology.orchestra.console.trial145:check power output: Chassis Power is on
2026-02-20T14:18:58.552 DEBUG:teuthology.orchestra.console.trial145:pexpect command: ipmitool -H trial145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-20T14:18:58.663 DEBUG:teuthology.orchestra.console.trial145:check power output: Chassis Power is off
2026-02-20T14:18:58.663 INFO:teuthology.orchestra.console.trial145:Power off completed
2026-02-20T14:18:58.876 INFO:teuthology.lock.ops:unlocked: trial145.front.sepia.ceph.com