2026-02-25T11:30:59.574 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-25T11:30:59.574 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:30:59.646 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:30:59.897 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x70aedf50cc20>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x70aedf4a0040>, signals=[15])
2026-02-25T11:30:59.897 INFO:teuthology.lock.ops:Start node 'trial028.front.sepia.ceph.com' reimaging
2026-02-25T11:30:59.897 INFO:teuthology.lock.ops:Updating [trial028.front.sepia.ceph.com]: reset os type and version on server
2026-02-25T11:30:59.897 INFO:teuthology.lock.ops:Updating trial028.front.sepia.ceph.com on lock server
2026-02-25T11:30:59.953 INFO:teuthology.lock.ops:Start node 'trial119.front.sepia.ceph.com' reimaging
2026-02-25T11:30:59.953 INFO:teuthology.lock.ops:Updating [trial119.front.sepia.ceph.com]: reset os type and version on server
2026-02-25T11:30:59.953 INFO:teuthology.lock.ops:Updating trial119.front.sepia.ceph.com on lock server
2026-02-25T11:31:00.011 INFO:teuthology.lock.ops:Start node 'trial167.front.sepia.ceph.com' reimaging
2026-02-25T11:31:00.011 INFO:teuthology.lock.ops:Updating [trial167.front.sepia.ceph.com]: reset os type and version on server
2026-02-25T11:31:00.011 INFO:teuthology.lock.ops:Updating trial167.front.sepia.ceph.com on lock server
2026-02-25T11:31:00.208 DEBUG:teuthology.provision.fog.trial028:Requesting image trial_centos_9.stream (ID 3)
2026-02-25T11:31:00.256 DEBUG:teuthology.provision.fog.trial119:Requesting image trial_centos_9.stream (ID 3)
2026-02-25T11:31:00.262 INFO:teuthology.provision.fog.trial028:Scheduling deploy of centos 9.stream
2026-02-25T11:31:00.310 INFO:teuthology.provision.fog.trial119:Scheduling deploy of centos 9.stream
2026-02-25T11:31:00.313 DEBUG:teuthology.provision.fog.trial167:Requesting image trial_centos_9.stream (ID 3)
2026-02-25T11:31:00.360 INFO:teuthology.provision.fog.trial167:Scheduling deploy of centos 9.stream
2026-02-25T11:31:01.983 INFO:teuthology.orchestra.console.trial028:Power off
2026-02-25T11:31:01.984 DEBUG:teuthology.orchestra.console.trial028:pexpect command: ipmitool -H trial028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-25T11:31:02.042 INFO:teuthology.orchestra.console.trial167:Power off
2026-02-25T11:31:02.042 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-25T11:31:02.048 DEBUG:teuthology.orchestra.console.trial028:power off output: Chassis Power Control: Down/Off
2026-02-25T11:31:02.063 INFO:teuthology.orchestra.console.trial119:Power off
2026-02-25T11:31:02.063 DEBUG:teuthology.orchestra.console.trial119:pexpect command: ipmitool -H trial119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-25T11:31:02.081 DEBUG:teuthology.orchestra.console.trial167:power off output: Chassis Power Control: Down/Off
2026-02-25T11:31:02.100 DEBUG:teuthology.orchestra.console.trial119:power off output: Chassis Power Control: Down/Off
2026-02-25T11:31:06.053 DEBUG:teuthology.orchestra.console.trial028:pexpect command: ipmitool -H trial028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T11:31:06.082 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T11:31:06.100 DEBUG:teuthology.orchestra.console.trial119:pexpect command: ipmitool -H trial119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T11:31:06.161 DEBUG:teuthology.orchestra.console.trial028:check power output: Chassis Power is off
2026-02-25T11:31:06.162 INFO:teuthology.orchestra.console.trial028:Power off completed
2026-02-25T11:31:06.190 DEBUG:teuthology.orchestra.console.trial167:check power output: Chassis Power is off
2026-02-25T11:31:06.190 INFO:teuthology.orchestra.console.trial167:Power off completed
2026-02-25T11:31:06.210 DEBUG:teuthology.orchestra.console.trial119:check power output: Chassis Power is off
2026-02-25T11:31:06.210 INFO:teuthology.orchestra.console.trial119:Power off completed
2026-02-25T11:31:06.262 INFO:teuthology.orchestra.console.trial028:Power on
2026-02-25T11:31:06.263 DEBUG:teuthology.orchestra.console.trial028:pexpect command: ipmitool -H trial028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-25T11:31:06.291 INFO:teuthology.orchestra.console.trial167:Power on
2026-02-25T11:31:06.291 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-25T11:31:06.307 DEBUG:teuthology.orchestra.console.trial028:power on output: Chassis Power Control: Up/On
2026-02-25T11:31:06.311 INFO:teuthology.orchestra.console.trial119:Power on
2026-02-25T11:31:06.311 DEBUG:teuthology.orchestra.console.trial119:pexpect command: ipmitool -H trial119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-25T11:31:06.331 DEBUG:teuthology.orchestra.console.trial167:power on output: Chassis Power Control: Up/On
2026-02-25T11:31:06.350 DEBUG:teuthology.orchestra.console.trial119:power on output: Chassis Power Control: Up/On
2026-02-25T11:31:10.311 DEBUG:teuthology.orchestra.console.trial028:pexpect command: ipmitool -H trial028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T11:31:10.333 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T11:31:10.350 DEBUG:teuthology.orchestra.console.trial119:pexpect command: ipmitool -H trial119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T11:31:10.420 DEBUG:teuthology.orchestra.console.trial028:check power output: Chassis Power is off
2026-02-25T11:31:10.440 DEBUG:teuthology.orchestra.console.trial167:check power output: Chassis Power is on
2026-02-25T11:31:10.441 INFO:teuthology.orchestra.console.trial167:Power on completed
2026-02-25T11:31:10.458 DEBUG:teuthology.orchestra.console.trial119:check power output: Chassis Power is off
2026-02-25T11:31:10.542 INFO:teuthology.provision.fog.trial167:Waiting for deploy to finish
2026-02-25T11:31:14.425 DEBUG:teuthology.orchestra.console.trial028:pexpect command: ipmitool -H trial028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T11:31:14.460 DEBUG:teuthology.orchestra.console.trial119:pexpect command: ipmitool -H trial119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T11:31:14.533 DEBUG:teuthology.orchestra.console.trial028:check power output: Chassis Power is on
2026-02-25T11:31:14.534 INFO:teuthology.orchestra.console.trial028:Power on completed
2026-02-25T11:31:14.568 DEBUG:teuthology.orchestra.console.trial119:check power output: Chassis Power is on
2026-02-25T11:31:14.568 INFO:teuthology.orchestra.console.trial119:Power on completed
2026-02-25T11:31:14.635 INFO:teuthology.provision.fog.trial028:Waiting for deploy to finish
2026-02-25T11:31:14.669 INFO:teuthology.provision.fog.trial119:Waiting for deploy to finish
2026-02-25T11:32:58.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T11:33:02.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T11:33:02.717 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T11:33:05.792 WARNING:teuthology.provision.fog.trial119:[Errno None] Unable to connect to port 22 on 10.20.193.119
2026-02-25T11:33:05.792 WARNING:teuthology.provision.fog.trial028:[Errno None] Unable to connect to port 22 on 10.20.193.28
2026-02-25T11:33:11.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T11:33:11.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T11:33:14.880 WARNING:teuthology.provision.fog.trial119:[Errno None] Unable to connect to port 22 on 10.20.193.119
2026-02-25T11:33:14.880 WARNING:teuthology.provision.fog.trial028:[Errno None] Unable to connect to port 22 on 10.20.193.28
2026-02-25T11:33:20.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T11:33:20.888 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T11:33:23.967 WARNING:teuthology.provision.fog.trial119:[Errno None] Unable to connect to port 22 on 10.20.193.119
2026-02-25T11:33:23.968 WARNING:teuthology.provision.fog.trial028:[Errno None] Unable to connect to port 22 on 10.20.193.28
2026-02-25T11:33:29.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T11:33:29.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T11:33:33.056 WARNING:teuthology.provision.fog.trial119:[Errno None] Unable to connect to port 22 on 10.20.193.119
2026-02-25T11:33:33.056 WARNING:teuthology.provision.fog.trial028:[Errno None] Unable to connect to port 22 on 10.20.193.28
2026-02-25T11:33:39.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T11:33:39.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T11:33:42.144 WARNING:teuthology.provision.fog.trial119:[Errno None] Unable to connect to port 22 on 10.20.193.119
2026-02-25T11:33:42.144 WARNING:teuthology.provision.fog.trial028:[Errno None] Unable to connect to port 22 on 10.20.193.28
2026-02-25T11:33:48.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T11:33:48.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T11:33:58.291 WARNING:teuthology.provision.fog.trial167:timed out
2026-02-25T11:34:04.298 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T11:34:09.546 ERROR:teuthology.orchestra.connection:Error authenticating with trial167.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-25T11:34:13.647 ERROR:teuthology.orchestra.connection:Error authenticating with trial167.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-25T11:34:20.727 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-25T11:34:20.728 DEBUG:teuthology.orchestra.run.trial167:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-25T11:34:20.907 INFO:teuthology.provision.fog.trial167:Node is ready
2026-02-25T11:34:20.907 DEBUG:teuthology.orchestra.run.trial167:> hostname
2026-02-25T11:34:20.959 INFO:teuthology.orchestra.run.trial167.stdout:trial167.front.sepia.ceph.com
2026-02-25T11:34:20.960 DEBUG:teuthology.orchestra.run.trial167:> grep trial167.front.sepia.ceph.com /etc/hosts
2026-02-25T11:34:21.011 INFO:teuthology.orchestra.run.trial167.stdout:10.20.193.167 trial167.front.sepia.ceph.com trial167
2026-02-25T11:34:21.011 DEBUG:teuthology.orchestra.run.trial167:> sudo hostname trial167
2026-02-25T11:34:21.081 DEBUG:teuthology.orchestra.run.trial167:> sudo sed -i -e 's/trial167.front.sepia.ceph.com/trial167/g' /etc/hosts
2026-02-25T11:34:21.143 DEBUG:teuthology.orchestra.run.trial167:> sudo sed -i -e 's/10.20.193.167/10.20.193.167/g' /etc/hosts
2026-02-25T11:34:21.207 DEBUG:teuthology.orchestra.run.trial167:> sudo sed -i -e 's/trial167.front.sepia.ceph.com/trial167/g' /etc/hostname
2026-02-25T11:34:21.269 DEBUG:teuthology.orchestra.run.trial167:> sudo hostname trial167
2026-02-25T11:34:21.332 DEBUG:teuthology.orchestra.run.trial167:> cat /etc/os-release
2026-02-25T11:34:21.386 INFO:teuthology.orchestra.run.trial167.stdout:NAME="CentOS Stream"
2026-02-25T11:34:21.387 INFO:teuthology.orchestra.run.trial167.stdout:VERSION="9"
2026-02-25T11:34:21.387 INFO:teuthology.orchestra.run.trial167.stdout:ID="centos"
2026-02-25T11:34:21.387 INFO:teuthology.orchestra.run.trial167.stdout:ID_LIKE="rhel fedora"
2026-02-25T11:34:21.387 INFO:teuthology.orchestra.run.trial167.stdout:VERSION_ID="9"
2026-02-25T11:34:21.387 INFO:teuthology.orchestra.run.trial167.stdout:PLATFORM_ID="platform:el9"
2026-02-25T11:34:21.387 INFO:teuthology.orchestra.run.trial167.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-25T11:34:21.387 INFO:teuthology.orchestra.run.trial167.stdout:ANSI_COLOR="0;31"
2026-02-25T11:34:21.387 INFO:teuthology.orchestra.run.trial167.stdout:LOGO="fedora-logo-icon"
2026-02-25T11:34:21.387 INFO:teuthology.orchestra.run.trial167.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-25T11:34:21.388 INFO:teuthology.orchestra.run.trial167.stdout:HOME_URL="https://centos.org/"
2026-02-25T11:34:21.388 INFO:teuthology.orchestra.run.trial167.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-25T11:34:21.388 INFO:teuthology.orchestra.run.trial167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-25T11:34:21.388 INFO:teuthology.orchestra.run.trial167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-25T11:34:21.389 INFO:teuthology.provision.fog.trial167:Deploy complete!
2026-02-25T11:34:21.389 DEBUG:teuthology.parallel:result is None
2026-02-25T11:34:24.031 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-25T11:34:24.032 DEBUG:teuthology.orchestra.run.trial028:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-25T11:34:24.034 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-25T11:34:24.034 DEBUG:teuthology.orchestra.run.trial119:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-25T11:34:24.191 INFO:teuthology.provision.fog.trial028:Node is ready
2026-02-25T11:34:24.191 DEBUG:teuthology.orchestra.run.trial028:> hostname
2026-02-25T11:34:24.247 INFO:teuthology.orchestra.run.trial028.stdout:trial028.front.sepia.ceph.com
2026-02-25T11:34:24.247 DEBUG:teuthology.orchestra.run.trial028:> grep trial028.front.sepia.ceph.com /etc/hosts
2026-02-25T11:34:24.270 INFO:teuthology.provision.fog.trial119:Node is ready
2026-02-25T11:34:24.270 DEBUG:teuthology.orchestra.run.trial119:> hostname
2026-02-25T11:34:24.303 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28 trial028.front.sepia.ceph.com trial028
2026-02-25T11:34:24.304 DEBUG:teuthology.orchestra.run.trial028:> sudo hostname trial028
2026-02-25T11:34:24.327 INFO:teuthology.orchestra.run.trial119.stdout:trial119.front.sepia.ceph.com
2026-02-25T11:34:24.327 DEBUG:teuthology.orchestra.run.trial119:> grep trial119.front.sepia.ceph.com /etc/hosts
2026-02-25T11:34:24.372 DEBUG:teuthology.orchestra.run.trial028:> sudo sed -i -e 's/trial028.front.sepia.ceph.com/trial028/g' /etc/hosts
2026-02-25T11:34:24.385 INFO:teuthology.orchestra.run.trial119.stdout:10.20.193.119 trial119.front.sepia.ceph.com trial119
2026-02-25T11:34:24.385 DEBUG:teuthology.orchestra.run.trial119:> sudo hostname trial119
2026-02-25T11:34:24.435 DEBUG:teuthology.orchestra.run.trial028:> sudo sed -i -e 's/10.20.193.28/10.20.193.28/g' /etc/hosts
2026-02-25T11:34:24.455 DEBUG:teuthology.orchestra.run.trial119:> sudo sed -i -e 's/trial119.front.sepia.ceph.com/trial119/g' /etc/hosts
2026-02-25T11:34:24.500 DEBUG:teuthology.orchestra.run.trial028:> sudo sed -i -e 's/trial028.front.sepia.ceph.com/trial028/g' /etc/hostname
2026-02-25T11:34:24.522 DEBUG:teuthology.orchestra.run.trial119:> sudo sed -i -e 's/10.20.193.119/10.20.193.119/g' /etc/hosts
2026-02-25T11:34:24.564 DEBUG:teuthology.orchestra.run.trial028:> sudo hostname trial028
2026-02-25T11:34:24.607 DEBUG:teuthology.orchestra.run.trial119:> sudo sed -i -e 's/trial119.front.sepia.ceph.com/trial119/g' /etc/hostname
2026-02-25T11:34:24.627 DEBUG:teuthology.orchestra.run.trial028:> cat /etc/os-release
2026-02-25T11:34:24.633 DEBUG:teuthology.orchestra.run.trial119:> sudo hostname trial119
2026-02-25T11:34:24.684 INFO:teuthology.orchestra.run.trial028.stdout:NAME="CentOS Stream"
2026-02-25T11:34:24.684 INFO:teuthology.orchestra.run.trial028.stdout:VERSION="9"
2026-02-25T11:34:24.684 INFO:teuthology.orchestra.run.trial028.stdout:ID="centos"
2026-02-25T11:34:24.685 INFO:teuthology.orchestra.run.trial028.stdout:ID_LIKE="rhel fedora"
2026-02-25T11:34:24.685 INFO:teuthology.orchestra.run.trial028.stdout:VERSION_ID="9"
2026-02-25T11:34:24.685 INFO:teuthology.orchestra.run.trial028.stdout:PLATFORM_ID="platform:el9"
2026-02-25T11:34:24.685 INFO:teuthology.orchestra.run.trial028.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-25T11:34:24.685 INFO:teuthology.orchestra.run.trial028.stdout:ANSI_COLOR="0;31"
2026-02-25T11:34:24.685 INFO:teuthology.orchestra.run.trial028.stdout:LOGO="fedora-logo-icon"
2026-02-25T11:34:24.685 INFO:teuthology.orchestra.run.trial028.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-25T11:34:24.685 INFO:teuthology.orchestra.run.trial028.stdout:HOME_URL="https://centos.org/"
2026-02-25T11:34:24.685 INFO:teuthology.orchestra.run.trial028.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-25T11:34:24.685 INFO:teuthology.orchestra.run.trial028.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-25T11:34:24.685 INFO:teuthology.orchestra.run.trial028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-25T11:34:24.686 INFO:teuthology.provision.fog.trial028:Deploy complete!
2026-02-25T11:34:24.687 DEBUG:teuthology.parallel:result is None
2026-02-25T11:34:24.699 DEBUG:teuthology.orchestra.run.trial119:> cat /etc/os-release
2026-02-25T11:34:24.756 INFO:teuthology.orchestra.run.trial119.stdout:NAME="CentOS Stream"
2026-02-25T11:34:24.756 INFO:teuthology.orchestra.run.trial119.stdout:VERSION="9"
2026-02-25T11:34:24.757 INFO:teuthology.orchestra.run.trial119.stdout:ID="centos"
2026-02-25T11:34:24.757 INFO:teuthology.orchestra.run.trial119.stdout:ID_LIKE="rhel fedora"
2026-02-25T11:34:24.757 INFO:teuthology.orchestra.run.trial119.stdout:VERSION_ID="9"
2026-02-25T11:34:24.757 INFO:teuthology.orchestra.run.trial119.stdout:PLATFORM_ID="platform:el9"
2026-02-25T11:34:24.757 INFO:teuthology.orchestra.run.trial119.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-25T11:34:24.757 INFO:teuthology.orchestra.run.trial119.stdout:ANSI_COLOR="0;31"
2026-02-25T11:34:24.757 INFO:teuthology.orchestra.run.trial119.stdout:LOGO="fedora-logo-icon"
2026-02-25T11:34:24.757 INFO:teuthology.orchestra.run.trial119.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-25T11:34:24.758 INFO:teuthology.orchestra.run.trial119.stdout:HOME_URL="https://centos.org/"
2026-02-25T11:34:24.758 INFO:teuthology.orchestra.run.trial119.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-25T11:34:24.758 INFO:teuthology.orchestra.run.trial119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-25T11:34:24.758 INFO:teuthology.orchestra.run.trial119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-25T11:34:24.758 INFO:teuthology.provision.fog.trial119:Deploy complete!
2026-02-25T11:34:24.759 DEBUG:teuthology.parallel:result is None
2026-02-25T11:34:25.287 INFO:teuthology.lock.ops:Checking trial028.front.sepia.ceph.com
2026-02-25T11:34:25.288 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-25T11:34:25.390 INFO:teuthology.lock.ops:Checking trial119.front.sepia.ceph.com
2026-02-25T11:34:25.391 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-25T11:34:25.453 INFO:teuthology.lock.ops:Checking trial167.front.sepia.ceph.com
2026-02-25T11:34:25.453 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-25T11:34:25.510 INFO:teuthology.lock.ops:Updating [trial028.front.sepia.ceph.com]: set os type and version on server
2026-02-25T11:34:25.510 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial028.front.sepia.ceph.com'
2026-02-25T11:34:25.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T11:34:25.570 DEBUG:teuthology.orchestra.run.trial028:> true
2026-02-25T11:34:25.644 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial028.front.sepia.ceph.com'
2026-02-25T11:34:25.644 DEBUG:teuthology.orchestra.run.trial028:> uname -m
2026-02-25T11:34:25.699 INFO:teuthology.orchestra.run.trial028.stdout:x86_64
2026-02-25T11:34:25.699 DEBUG:teuthology.orchestra.run.trial028:> cat /etc/os-release
2026-02-25T11:34:25.756 INFO:teuthology.orchestra.run.trial028.stdout:NAME="CentOS Stream"
2026-02-25T11:34:25.756 INFO:teuthology.orchestra.run.trial028.stdout:VERSION="9"
2026-02-25T11:34:25.756 INFO:teuthology.orchestra.run.trial028.stdout:ID="centos"
2026-02-25T11:34:25.756 INFO:teuthology.orchestra.run.trial028.stdout:ID_LIKE="rhel fedora"
2026-02-25T11:34:25.756 INFO:teuthology.orchestra.run.trial028.stdout:VERSION_ID="9"
2026-02-25T11:34:25.757 INFO:teuthology.orchestra.run.trial028.stdout:PLATFORM_ID="platform:el9"
2026-02-25T11:34:25.757 INFO:teuthology.orchestra.run.trial028.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-25T11:34:25.757 INFO:teuthology.orchestra.run.trial028.stdout:ANSI_COLOR="0;31"
2026-02-25T11:34:25.757 INFO:teuthology.orchestra.run.trial028.stdout:LOGO="fedora-logo-icon"
2026-02-25T11:34:25.757 INFO:teuthology.orchestra.run.trial028.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-25T11:34:25.757 INFO:teuthology.orchestra.run.trial028.stdout:HOME_URL="https://centos.org/"
2026-02-25T11:34:25.757 INFO:teuthology.orchestra.run.trial028.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-25T11:34:25.757 INFO:teuthology.orchestra.run.trial028.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-25T11:34:25.757 INFO:teuthology.orchestra.run.trial028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-25T11:34:25.758 INFO:teuthology.lock.ops:Updating trial028.front.sepia.ceph.com on lock server
2026-02-25T11:34:25.860 INFO:teuthology.lock.ops:Updating [trial119.front.sepia.ceph.com]: set os type and version on server
2026-02-25T11:34:25.860 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial119.front.sepia.ceph.com'
2026-02-25T11:34:25.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T11:34:25.923 DEBUG:teuthology.orchestra.run.trial119:> true
2026-02-25T11:34:25.997 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial119.front.sepia.ceph.com'
2026-02-25T11:34:25.997 DEBUG:teuthology.orchestra.run.trial119:> uname -m
2026-02-25T11:34:26.052 INFO:teuthology.orchestra.run.trial119.stdout:x86_64
2026-02-25T11:34:26.053 DEBUG:teuthology.orchestra.run.trial119:> cat /etc/os-release
2026-02-25T11:34:26.109 INFO:teuthology.orchestra.run.trial119.stdout:NAME="CentOS Stream"
2026-02-25T11:34:26.109 INFO:teuthology.orchestra.run.trial119.stdout:VERSION="9"
2026-02-25T11:34:26.109 INFO:teuthology.orchestra.run.trial119.stdout:ID="centos"
2026-02-25T11:34:26.109 INFO:teuthology.orchestra.run.trial119.stdout:ID_LIKE="rhel fedora"
2026-02-25T11:34:26.109 INFO:teuthology.orchestra.run.trial119.stdout:VERSION_ID="9"
2026-02-25T11:34:26.109 INFO:teuthology.orchestra.run.trial119.stdout:PLATFORM_ID="platform:el9"
2026-02-25T11:34:26.109 INFO:teuthology.orchestra.run.trial119.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-25T11:34:26.110 INFO:teuthology.orchestra.run.trial119.stdout:ANSI_COLOR="0;31"
2026-02-25T11:34:26.110 INFO:teuthology.orchestra.run.trial119.stdout:LOGO="fedora-logo-icon"
2026-02-25T11:34:26.110 INFO:teuthology.orchestra.run.trial119.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-25T11:34:26.110 INFO:teuthology.orchestra.run.trial119.stdout:HOME_URL="https://centos.org/"
2026-02-25T11:34:26.110 INFO:teuthology.orchestra.run.trial119.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-25T11:34:26.110 INFO:teuthology.orchestra.run.trial119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-25T11:34:26.110 INFO:teuthology.orchestra.run.trial119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-25T11:34:26.111 INFO:teuthology.lock.ops:Updating trial119.front.sepia.ceph.com on lock server
2026-02-25T11:34:26.212 INFO:teuthology.lock.ops:Updating [trial167.front.sepia.ceph.com]: set os type and version on server
2026-02-25T11:34:26.212 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial167.front.sepia.ceph.com'
2026-02-25T11:34:26.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T11:34:26.288 DEBUG:teuthology.orchestra.run.trial167:> true
2026-02-25T11:34:26.360 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial167.front.sepia.ceph.com'
2026-02-25T11:34:26.360 DEBUG:teuthology.orchestra.run.trial167:> uname -m
2026-02-25T11:34:26.415 INFO:teuthology.orchestra.run.trial167.stdout:x86_64
2026-02-25T11:34:26.416 DEBUG:teuthology.orchestra.run.trial167:> cat /etc/os-release
2026-02-25T11:34:26.472 INFO:teuthology.orchestra.run.trial167.stdout:NAME="CentOS Stream"
2026-02-25T11:34:26.472 INFO:teuthology.orchestra.run.trial167.stdout:VERSION="9"
2026-02-25T11:34:26.472 INFO:teuthology.orchestra.run.trial167.stdout:ID="centos"
2026-02-25T11:34:26.472 INFO:teuthology.orchestra.run.trial167.stdout:ID_LIKE="rhel fedora"
2026-02-25T11:34:26.472 INFO:teuthology.orchestra.run.trial167.stdout:VERSION_ID="9"
2026-02-25T11:34:26.472 INFO:teuthology.orchestra.run.trial167.stdout:PLATFORM_ID="platform:el9"
2026-02-25T11:34:26.472 INFO:teuthology.orchestra.run.trial167.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-25T11:34:26.473 INFO:teuthology.orchestra.run.trial167.stdout:ANSI_COLOR="0;31"
2026-02-25T11:34:26.473 INFO:teuthology.orchestra.run.trial167.stdout:LOGO="fedora-logo-icon"
2026-02-25T11:34:26.473 INFO:teuthology.orchestra.run.trial167.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-25T11:34:26.473 INFO:teuthology.orchestra.run.trial167.stdout:HOME_URL="https://centos.org/"
2026-02-25T11:34:26.473 INFO:teuthology.orchestra.run.trial167.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-25T11:34:26.473 INFO:teuthology.orchestra.run.trial167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-25T11:34:26.473 INFO:teuthology.orchestra.run.trial167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-25T11:34:26.474 INFO:teuthology.lock.ops:Updating trial167.front.sepia.ceph.com on lock server
2026-02-25T11:34:26.576 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:34:26.665 INFO:teuthology.dispatcher.supervisor:Running job 69481
2026-02-25T11:34:26.665 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/bin/teuthology -v --owner scheduled_vshankar@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69481 --name vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/fs/test_o_trunc}} -- /home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69481/orig.config.yaml
2026-02-25T11:34:26.665 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69481
2026-02-25T11:34:26.665 INFO:teuthology.dispatcher.supervisor:Job PID: 602409
2026-02-25T11:34:26.665 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-25T11:36:26.766 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:38:26.917 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:40:27.119 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:42:27.315 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:44:27.503 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:46:27.687 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:48:27.880 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:50:28.068 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:52:28.228 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:54:28.458 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:56:28.645 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:58:28.853 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T12:00:29.031 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T12:02:29.229 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T12:02:29.304 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-25T12:02:29.474 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-25T12:02:30.221 INFO:teuthology.orchestra.console.trial028:Power off
2026-02-25T12:02:30.221 DEBUG:teuthology.orchestra.console.trial028:pexpect command: ipmitool -H trial028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-25T12:02:30.235 INFO:teuthology.orchestra.console.trial167:Power off
2026-02-25T12:02:30.235 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-25T12:02:30.242 INFO:teuthology.orchestra.console.trial119:Power off
2026-02-25T12:02:30.242 DEBUG:teuthology.orchestra.console.trial119:pexpect command: ipmitool -H trial119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-25T12:02:30.260 DEBUG:teuthology.orchestra.console.trial028:power off output: Chassis Power Control: Down/Off
2026-02-25T12:02:30.273 DEBUG:teuthology.orchestra.console.trial167:power off output: Chassis Power Control: Down/Off
2026-02-25T12:02:30.280 DEBUG:teuthology.orchestra.console.trial119:power off output: Chassis Power Control: Down/Off
2026-02-25T12:02:34.264 DEBUG:teuthology.orchestra.console.trial028:pexpect command: ipmitool -H trial028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T12:02:34.274 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T12:02:34.281 DEBUG:teuthology.orchestra.console.trial119:pexpect command: ipmitool -H trial119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T12:02:34.375 DEBUG:teuthology.orchestra.console.trial028:check power output: Chassis Power is on
2026-02-25T12:02:34.382 DEBUG:teuthology.orchestra.console.trial167:check power output: Chassis Power is on
2026-02-25T12:02:34.390 DEBUG:teuthology.orchestra.console.trial119:check power output: Chassis Power is on
2026-02-25T12:02:38.379 DEBUG:teuthology.orchestra.console.trial028:pexpect command: ipmitool -H trial028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T12:02:38.389 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T12:02:38.397 DEBUG:teuthology.orchestra.console.trial119:pexpect command: ipmitool -H trial119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T12:02:38.490 DEBUG:teuthology.orchestra.console.trial028:check power output: Chassis Power is off
2026-02-25T12:02:38.491 INFO:teuthology.orchestra.console.trial028:Power off completed
2026-02-25T12:02:38.499 DEBUG:teuthology.orchestra.console.trial167:check power output: Chassis Power is off
2026-02-25T12:02:38.499 INFO:teuthology.orchestra.console.trial167:Power off completed
2026-02-25T12:02:38.507 DEBUG:teuthology.orchestra.console.trial119:check power output: Chassis Power is off
2026-02-25T12:02:38.508 INFO:teuthology.orchestra.console.trial119:Power off completed
2026-02-25T12:02:38.804 INFO:teuthology.lock.ops:unlocked: trial028.front.sepia.ceph.com
2026-02-25T12:02:38.820 INFO:teuthology.lock.ops:unlocked: trial119.front.sepia.ceph.com
2026-02-25T12:02:38.827 INFO:teuthology.lock.ops:unlocked: trial167.front.sepia.ceph.com