2026-02-14T01:01:34.132 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71
2026-02-14T01:01:34.132 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:01:34.203 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:01:34.452 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x749dc1cfdee0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x749dc14953a0>, signals=[15])
2026-02-14T01:01:34.453 INFO:teuthology.lock.ops:Start node 'trial028.front.sepia.ceph.com' reimaging
2026-02-14T01:01:34.453 INFO:teuthology.lock.ops:Updating [trial028.front.sepia.ceph.com]: reset os type and version on server
2026-02-14T01:01:34.453 INFO:teuthology.lock.ops:Updating trial028.front.sepia.ceph.com on lock server
2026-02-14T01:01:34.510 INFO:teuthology.lock.ops:Start node 'trial153.front.sepia.ceph.com' reimaging
2026-02-14T01:01:34.510 INFO:teuthology.lock.ops:Updating [trial153.front.sepia.ceph.com]: reset os type and version on server
2026-02-14T01:01:34.510 INFO:teuthology.lock.ops:Updating trial153.front.sepia.ceph.com on lock server
2026-02-14T01:01:34.569 INFO:teuthology.lock.ops:Start node 'trial156.front.sepia.ceph.com' reimaging
2026-02-14T01:01:34.569 INFO:teuthology.lock.ops:Updating [trial156.front.sepia.ceph.com]: reset os type and version on server
2026-02-14T01:01:34.569 INFO:teuthology.lock.ops:Updating trial156.front.sepia.ceph.com on lock server
2026-02-14T01:01:34.801 DEBUG:teuthology.provision.fog.trial153:Requesting image trial_centos_9.stream (ID 3)
2026-02-14T01:01:34.801 DEBUG:teuthology.provision.fog.trial028:Requesting image trial_centos_9.stream (ID 3)
2026-02-14T01:01:34.827 INFO:teuthology.provision.fog.trial153:Scheduling deploy of centos 9.stream
2026-02-14T01:01:34.851 INFO:teuthology.provision.fog.trial028:Scheduling deploy of centos 9.stream
2026-02-14T01:01:34.855 DEBUG:teuthology.provision.fog.trial156:Requesting image trial_centos_9.stream (ID 3)
2026-02-14T01:01:34.908 INFO:teuthology.provision.fog.trial156:Scheduling deploy of centos 9.stream
2026-02-14T01:01:35.949 INFO:teuthology.orchestra.console.trial153:Power off
2026-02-14T01:01:35.949 DEBUG:teuthology.orchestra.console.trial153:pexpect command: ipmitool -H trial153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T01:01:36.010 INFO:teuthology.orchestra.console.trial028:Power off
2026-02-14T01:01:36.010 DEBUG:teuthology.orchestra.console.trial028:pexpect command: ipmitool -H trial028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T01:01:36.016 DEBUG:teuthology.orchestra.console.trial153:power off output: Chassis Power Control: Down/Off
2026-02-14T01:01:36.029 INFO:teuthology.orchestra.console.trial156:Power off
2026-02-14T01:01:36.029 DEBUG:teuthology.orchestra.console.trial156:pexpect command: ipmitool -H trial156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T01:01:36.048 DEBUG:teuthology.orchestra.console.trial028:power off output: Chassis Power Control: Down/Off
2026-02-14T01:01:36.074 DEBUG:teuthology.orchestra.console.trial156:power off output: Chassis Power Control: Down/Off
2026-02-14T01:01:40.021 DEBUG:teuthology.orchestra.console.trial153:pexpect command: ipmitool -H trial153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:01:40.049 DEBUG:teuthology.orchestra.console.trial028:pexpect command: ipmitool -H trial028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:01:40.075 DEBUG:teuthology.orchestra.console.trial156:pexpect command: ipmitool -H trial156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:01:40.131 DEBUG:teuthology.orchestra.console.trial153:check power output: Chassis Power is off
2026-02-14T01:01:40.131 INFO:teuthology.orchestra.console.trial153:Power off completed
2026-02-14T01:01:40.157 DEBUG:teuthology.orchestra.console.trial028:check power output: Chassis Power is off
2026-02-14T01:01:40.158 INFO:teuthology.orchestra.console.trial028:Power off completed
2026-02-14T01:01:40.184 DEBUG:teuthology.orchestra.console.trial156:check power output: Chassis Power is off
2026-02-14T01:01:40.185 INFO:teuthology.orchestra.console.trial156:Power off completed
2026-02-14T01:01:40.232 INFO:teuthology.orchestra.console.trial153:Power on
2026-02-14T01:01:40.233 DEBUG:teuthology.orchestra.console.trial153:pexpect command: ipmitool -H trial153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-14T01:01:40.259 INFO:teuthology.orchestra.console.trial028:Power on
2026-02-14T01:01:40.260 DEBUG:teuthology.orchestra.console.trial028:pexpect command: ipmitool -H trial028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-14T01:01:40.274 DEBUG:teuthology.orchestra.console.trial153:power on output: Chassis Power Control: Up/On
2026-02-14T01:01:40.286 INFO:teuthology.orchestra.console.trial156:Power on
2026-02-14T01:01:40.286 DEBUG:teuthology.orchestra.console.trial156:pexpect command: ipmitool -H trial156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-14T01:01:40.299 DEBUG:teuthology.orchestra.console.trial028:power on output: Chassis Power Control: Up/On
2026-02-14T01:01:40.326 DEBUG:teuthology.orchestra.console.trial156:power on output: Chassis Power Control: Up/On
2026-02-14T01:01:44.279 DEBUG:teuthology.orchestra.console.trial153:pexpect command: ipmitool -H trial153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:01:44.301 DEBUG:teuthology.orchestra.console.trial028:pexpect command: ipmitool -H trial028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:01:44.327 DEBUG:teuthology.orchestra.console.trial156:pexpect command: ipmitool -H trial156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:01:44.390 DEBUG:teuthology.orchestra.console.trial153:check power output: Chassis Power is on
2026-02-14T01:01:44.390 INFO:teuthology.orchestra.console.trial153:Power on completed
2026-02-14T01:01:44.410 DEBUG:teuthology.orchestra.console.trial028:check power output: Chassis Power is on
2026-02-14T01:01:44.410 INFO:teuthology.orchestra.console.trial028:Power on completed
2026-02-14T01:01:44.436 DEBUG:teuthology.orchestra.console.trial156:check power output: Chassis Power is on
2026-02-14T01:01:44.436 INFO:teuthology.orchestra.console.trial156:Power on completed
2026-02-14T01:01:44.491 INFO:teuthology.provision.fog.trial153:Waiting for deploy to finish
2026-02-14T01:01:44.511 INFO:teuthology.provision.fog.trial028:Waiting for deploy to finish
2026-02-14T01:01:44.537 INFO:teuthology.provision.fog.trial156:Waiting for deploy to finish
2026-02-14T01:03:31.854 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:03:31.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:03:31.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:03:31.955 ERROR:teuthology.orchestra.connection:Error authenticating with trial153.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2026-02-14T01:03:32.041 ERROR:teuthology.orchestra.connection:Error authenticating with trial028.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2026-02-14T01:04:31.949 WARNING:teuthology.provision.fog.trial156:timed out
2026-02-14T01:04:35.957 WARNING:teuthology.provision.fog.trial153:timed out
2026-02-14T01:04:36.043 WARNING:teuthology.provision.fog.trial028:timed out
2026-02-14T01:04:37.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:04:41.958 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:04:42.045 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:04:46.132 ERROR:teuthology.orchestra.connection:Error authenticating with trial153.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-14T01:04:49.343 ERROR:teuthology.orchestra.connection:Error authenticating with trial156.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-14T01:04:49.372 ERROR:teuthology.orchestra.connection:Error authenticating with trial028.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-14T01:04:50.258 ERROR:teuthology.orchestra.connection:Error authenticating with trial153.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-14T01:04:53.447 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-14T01:04:53.447 DEBUG:teuthology.orchestra.run.trial156:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-14T01:04:53.457 ERROR:teuthology.orchestra.connection:Error authenticating with trial028.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-14T01:04:53.630 INFO:teuthology.provision.fog.trial156:Node is ready
2026-02-14T01:04:53.630 DEBUG:teuthology.orchestra.run.trial156:> hostname
2026-02-14T01:04:53.683 INFO:teuthology.orchestra.run.trial156.stdout:trial156.front.sepia.ceph.com
2026-02-14T01:04:53.683 DEBUG:teuthology.orchestra.run.trial156:> grep trial156.front.sepia.ceph.com /etc/hosts
2026-02-14T01:04:53.736 INFO:teuthology.orchestra.run.trial156.stdout:10.20.193.156 trial156.front.sepia.ceph.com trial156
2026-02-14T01:04:53.736 DEBUG:teuthology.orchestra.run.trial156:> sudo hostname trial156
2026-02-14T01:04:53.803 DEBUG:teuthology.orchestra.run.trial156:> sudo sed -i -e 's/trial156.front.sepia.ceph.com/trial156/g' /etc/hosts
2026-02-14T01:04:53.866 DEBUG:teuthology.orchestra.run.trial156:> sudo sed -i -e 's/10.20.193.156/10.20.193.156/g' /etc/hosts
2026-02-14T01:04:53.928 DEBUG:teuthology.orchestra.run.trial156:> sudo sed -i -e 's/trial156.front.sepia.ceph.com/trial156/g' /etc/hostname
2026-02-14T01:04:53.991 DEBUG:teuthology.orchestra.run.trial156:> sudo hostname trial156
2026-02-14T01:04:54.053 DEBUG:teuthology.orchestra.run.trial156:> cat /etc/os-release
2026-02-14T01:04:54.106 INFO:teuthology.orchestra.run.trial156.stdout:NAME="CentOS Stream"
2026-02-14T01:04:54.106 INFO:teuthology.orchestra.run.trial156.stdout:VERSION="9"
2026-02-14T01:04:54.106 INFO:teuthology.orchestra.run.trial156.stdout:ID="centos"
2026-02-14T01:04:54.106 INFO:teuthology.orchestra.run.trial156.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:04:54.107 INFO:teuthology.orchestra.run.trial156.stdout:VERSION_ID="9"
2026-02-14T01:04:54.107 INFO:teuthology.orchestra.run.trial156.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:04:54.107 INFO:teuthology.orchestra.run.trial156.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:04:54.107 INFO:teuthology.orchestra.run.trial156.stdout:ANSI_COLOR="0;31"
2026-02-14T01:04:54.107 INFO:teuthology.orchestra.run.trial156.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:04:54.107 INFO:teuthology.orchestra.run.trial156.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:04:54.107 INFO:teuthology.orchestra.run.trial156.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:04:54.107 INFO:teuthology.orchestra.run.trial156.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:04:54.107 INFO:teuthology.orchestra.run.trial156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:04:54.107 INFO:teuthology.orchestra.run.trial156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:04:54.107 INFO:teuthology.provision.fog.trial156:Deploy complete!
2026-02-14T01:04:54.108 DEBUG:teuthology.parallel:result is None
2026-02-14T01:04:57.373 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-14T01:04:57.373 DEBUG:teuthology.orchestra.run.trial153:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-14T01:04:57.576 INFO:teuthology.provision.fog.trial153:Node is ready
2026-02-14T01:04:57.576 DEBUG:teuthology.orchestra.run.trial153:> hostname
2026-02-14T01:04:57.632 INFO:teuthology.orchestra.run.trial153.stdout:trial153.front.sepia.ceph.com
2026-02-14T01:04:57.633 DEBUG:teuthology.orchestra.run.trial153:> grep trial153.front.sepia.ceph.com /etc/hosts
2026-02-14T01:04:57.688 INFO:teuthology.orchestra.run.trial153.stdout:10.20.193.153 trial153.front.sepia.ceph.com trial153
2026-02-14T01:04:57.688 DEBUG:teuthology.orchestra.run.trial153:> sudo hostname trial153
2026-02-14T01:04:57.758 DEBUG:teuthology.orchestra.run.trial153:> sudo sed -i -e 's/trial153.front.sepia.ceph.com/trial153/g' /etc/hosts
2026-02-14T01:04:57.823 DEBUG:teuthology.orchestra.run.trial153:> sudo sed -i -e 's/10.20.193.153/10.20.193.153/g' /etc/hosts
2026-02-14T01:04:57.888 DEBUG:teuthology.orchestra.run.trial153:> sudo sed -i -e 's/trial153.front.sepia.ceph.com/trial153/g' /etc/hostname
2026-02-14T01:04:57.953 DEBUG:teuthology.orchestra.run.trial153:> sudo hostname trial153
2026-02-14T01:04:58.017 DEBUG:teuthology.orchestra.run.trial153:> cat /etc/os-release
2026-02-14T01:04:58.073 INFO:teuthology.orchestra.run.trial153.stdout:NAME="CentOS Stream"
2026-02-14T01:04:58.073 INFO:teuthology.orchestra.run.trial153.stdout:VERSION="9"
2026-02-14T01:04:58.074 INFO:teuthology.orchestra.run.trial153.stdout:ID="centos"
2026-02-14T01:04:58.074 INFO:teuthology.orchestra.run.trial153.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:04:58.074 INFO:teuthology.orchestra.run.trial153.stdout:VERSION_ID="9"
2026-02-14T01:04:58.074 INFO:teuthology.orchestra.run.trial153.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:04:58.074 INFO:teuthology.orchestra.run.trial153.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:04:58.074 INFO:teuthology.orchestra.run.trial153.stdout:ANSI_COLOR="0;31"
2026-02-14T01:04:58.074 INFO:teuthology.orchestra.run.trial153.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:04:58.074 INFO:teuthology.orchestra.run.trial153.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:04:58.074 INFO:teuthology.orchestra.run.trial153.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:04:58.074 INFO:teuthology.orchestra.run.trial153.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:04:58.074 INFO:teuthology.orchestra.run.trial153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:04:58.075 INFO:teuthology.orchestra.run.trial153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:04:58.075 INFO:teuthology.provision.fog.trial153:Deploy complete!
2026-02-14T01:04:58.076 DEBUG:teuthology.parallel:result is None
2026-02-14T01:05:00.565 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-14T01:05:00.565 DEBUG:teuthology.orchestra.run.trial028:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-14T01:05:00.722 INFO:teuthology.provision.fog.trial028:Node is ready
2026-02-14T01:05:00.722 DEBUG:teuthology.orchestra.run.trial028:> hostname
2026-02-14T01:05:00.777 INFO:teuthology.orchestra.run.trial028.stdout:trial028.front.sepia.ceph.com
2026-02-14T01:05:00.778 DEBUG:teuthology.orchestra.run.trial028:> grep trial028.front.sepia.ceph.com /etc/hosts
2026-02-14T01:05:00.834 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28 trial028.front.sepia.ceph.com trial028
2026-02-14T01:05:00.835 DEBUG:teuthology.orchestra.run.trial028:> sudo hostname trial028
2026-02-14T01:05:00.903 DEBUG:teuthology.orchestra.run.trial028:> sudo sed -i -e 's/trial028.front.sepia.ceph.com/trial028/g' /etc/hosts
2026-02-14T01:05:00.967 DEBUG:teuthology.orchestra.run.trial028:> sudo sed -i -e 's/10.20.193.28/10.20.193.28/g' /etc/hosts
2026-02-14T01:05:01.032 DEBUG:teuthology.orchestra.run.trial028:> sudo sed -i -e 's/trial028.front.sepia.ceph.com/trial028/g' /etc/hostname
2026-02-14T01:05:01.096 DEBUG:teuthology.orchestra.run.trial028:> sudo hostname trial028
2026-02-14T01:05:01.160 DEBUG:teuthology.orchestra.run.trial028:> cat /etc/os-release
2026-02-14T01:05:01.217 INFO:teuthology.orchestra.run.trial028.stdout:NAME="CentOS Stream"
2026-02-14T01:05:01.217 INFO:teuthology.orchestra.run.trial028.stdout:VERSION="9"
2026-02-14T01:05:01.217 INFO:teuthology.orchestra.run.trial028.stdout:ID="centos"
2026-02-14T01:05:01.217 INFO:teuthology.orchestra.run.trial028.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:05:01.217 INFO:teuthology.orchestra.run.trial028.stdout:VERSION_ID="9"
2026-02-14T01:05:01.217 INFO:teuthology.orchestra.run.trial028.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:05:01.217 INFO:teuthology.orchestra.run.trial028.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:05:01.218 INFO:teuthology.orchestra.run.trial028.stdout:ANSI_COLOR="0;31"
2026-02-14T01:05:01.218 INFO:teuthology.orchestra.run.trial028.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:05:01.218 INFO:teuthology.orchestra.run.trial028.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:05:01.218 INFO:teuthology.orchestra.run.trial028.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:05:01.218 INFO:teuthology.orchestra.run.trial028.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:05:01.218 INFO:teuthology.orchestra.run.trial028.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:05:01.218 INFO:teuthology.orchestra.run.trial028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:05:01.219 INFO:teuthology.provision.fog.trial028:Deploy complete!
2026-02-14T01:05:01.219 DEBUG:teuthology.parallel:result is None
2026-02-14T01:05:01.741 INFO:teuthology.lock.ops:Checking trial028.front.sepia.ceph.com
2026-02-14T01:05:01.742 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-14T01:05:01.842 INFO:teuthology.lock.ops:Checking trial153.front.sepia.ceph.com
2026-02-14T01:05:01.842 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-14T01:05:01.906 INFO:teuthology.lock.ops:Checking trial156.front.sepia.ceph.com
2026-02-14T01:05:01.906 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-14T01:05:01.964 INFO:teuthology.lock.ops:Updating [trial028.front.sepia.ceph.com]: set os type and version on server
2026-02-14T01:05:01.965 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial028.front.sepia.ceph.com'
2026-02-14T01:05:01.965 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:05:02.024 DEBUG:teuthology.orchestra.run.trial028:> true
2026-02-14T01:05:02.096 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial028.front.sepia.ceph.com'
2026-02-14T01:05:02.097 DEBUG:teuthology.orchestra.run.trial028:> uname -m
2026-02-14T01:05:02.152 INFO:teuthology.orchestra.run.trial028.stdout:x86_64
2026-02-14T01:05:02.153 DEBUG:teuthology.orchestra.run.trial028:> cat /etc/os-release
2026-02-14T01:05:02.209 INFO:teuthology.orchestra.run.trial028.stdout:NAME="CentOS Stream"
2026-02-14T01:05:02.209 INFO:teuthology.orchestra.run.trial028.stdout:VERSION="9"
2026-02-14T01:05:02.209 INFO:teuthology.orchestra.run.trial028.stdout:ID="centos"
2026-02-14T01:05:02.209 INFO:teuthology.orchestra.run.trial028.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:05:02.209 INFO:teuthology.orchestra.run.trial028.stdout:VERSION_ID="9"
2026-02-14T01:05:02.209 INFO:teuthology.orchestra.run.trial028.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:05:02.209 INFO:teuthology.orchestra.run.trial028.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:05:02.209 INFO:teuthology.orchestra.run.trial028.stdout:ANSI_COLOR="0;31"
2026-02-14T01:05:02.209 INFO:teuthology.orchestra.run.trial028.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:05:02.210 INFO:teuthology.orchestra.run.trial028.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:05:02.210 INFO:teuthology.orchestra.run.trial028.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:05:02.210 INFO:teuthology.orchestra.run.trial028.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:05:02.210 INFO:teuthology.orchestra.run.trial028.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:05:02.210 INFO:teuthology.orchestra.run.trial028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:05:02.211 INFO:teuthology.lock.ops:Updating trial028.front.sepia.ceph.com on lock server
2026-02-14T01:05:02.313 INFO:teuthology.lock.ops:Updating [trial153.front.sepia.ceph.com]: set os type and version on server
2026-02-14T01:05:02.313 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial153.front.sepia.ceph.com'
2026-02-14T01:05:02.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:05:02.377 DEBUG:teuthology.orchestra.run.trial153:> true
2026-02-14T01:05:02.447 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial153.front.sepia.ceph.com'
2026-02-14T01:05:02.447 DEBUG:teuthology.orchestra.run.trial153:> uname -m
2026-02-14T01:05:02.503 INFO:teuthology.orchestra.run.trial153.stdout:x86_64
2026-02-14T01:05:02.503 DEBUG:teuthology.orchestra.run.trial153:> cat /etc/os-release
2026-02-14T01:05:02.559 INFO:teuthology.orchestra.run.trial153.stdout:NAME="CentOS Stream"
2026-02-14T01:05:02.559 INFO:teuthology.orchestra.run.trial153.stdout:VERSION="9"
2026-02-14T01:05:02.559 INFO:teuthology.orchestra.run.trial153.stdout:ID="centos"
2026-02-14T01:05:02.559 INFO:teuthology.orchestra.run.trial153.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:05:02.560 INFO:teuthology.orchestra.run.trial153.stdout:VERSION_ID="9"
2026-02-14T01:05:02.560 INFO:teuthology.orchestra.run.trial153.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:05:02.560 INFO:teuthology.orchestra.run.trial153.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:05:02.560 INFO:teuthology.orchestra.run.trial153.stdout:ANSI_COLOR="0;31"
2026-02-14T01:05:02.560 INFO:teuthology.orchestra.run.trial153.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:05:02.560 INFO:teuthology.orchestra.run.trial153.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:05:02.560 INFO:teuthology.orchestra.run.trial153.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:05:02.560 INFO:teuthology.orchestra.run.trial153.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:05:02.560 INFO:teuthology.orchestra.run.trial153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:05:02.560 INFO:teuthology.orchestra.run.trial153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:05:02.561 INFO:teuthology.lock.ops:Updating trial153.front.sepia.ceph.com on lock server
2026-02-14T01:05:02.664 INFO:teuthology.lock.ops:Updating [trial156.front.sepia.ceph.com]: set os type and version on server
2026-02-14T01:05:02.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial156.front.sepia.ceph.com'
2026-02-14T01:05:02.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:05:02.735 DEBUG:teuthology.orchestra.run.trial156:> true
2026-02-14T01:05:02.814 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial156.front.sepia.ceph.com'
2026-02-14T01:05:02.815 DEBUG:teuthology.orchestra.run.trial156:> uname -m
2026-02-14T01:05:02.871 INFO:teuthology.orchestra.run.trial156.stdout:x86_64
2026-02-14T01:05:02.871 DEBUG:teuthology.orchestra.run.trial156:> cat /etc/os-release
2026-02-14T01:05:02.927 INFO:teuthology.orchestra.run.trial156.stdout:NAME="CentOS Stream"
2026-02-14T01:05:02.927 INFO:teuthology.orchestra.run.trial156.stdout:VERSION="9"
2026-02-14T01:05:02.928 INFO:teuthology.orchestra.run.trial156.stdout:ID="centos"
2026-02-14T01:05:02.928 INFO:teuthology.orchestra.run.trial156.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:05:02.928 INFO:teuthology.orchestra.run.trial156.stdout:VERSION_ID="9"
2026-02-14T01:05:02.928 INFO:teuthology.orchestra.run.trial156.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:05:02.928 INFO:teuthology.orchestra.run.trial156.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:05:02.928 INFO:teuthology.orchestra.run.trial156.stdout:ANSI_COLOR="0;31"
2026-02-14T01:05:02.928 INFO:teuthology.orchestra.run.trial156.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:05:02.928 INFO:teuthology.orchestra.run.trial156.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:05:02.928 INFO:teuthology.orchestra.run.trial156.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:05:02.928 INFO:teuthology.orchestra.run.trial156.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:05:02.928 INFO:teuthology.orchestra.run.trial156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:05:02.929 INFO:teuthology.orchestra.run.trial156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:05:02.929 INFO:teuthology.lock.ops:Updating trial156.front.sepia.ceph.com on lock server
2026-02-14T01:05:03.032 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:05:03.136 INFO:teuthology.dispatcher.supervisor:Running job 50513
2026-02-14T01:05:03.136 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/bin/teuthology -v --owner scheduled_jcollin@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/jcollin-2026-02-14_00:12:18-fs-wip-jcollin-testing-20260213.113451-tentacle-distro-default-trial/50513 --name jcollin-2026-02-14_00:12:18-fs-wip-jcollin-testing-20260213.113451-tentacle-distro-default-trial --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/misc}} -- /home/teuthworker/mnt/teuthology/jcollin-2026-02-14_00:12:18-fs-wip-jcollin-testing-20260213.113451-tentacle-distro-default-trial/50513/orig.config.yaml
2026-02-14T01:05:03.137 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/jcollin-2026-02-14_00:12:18-fs-wip-jcollin-testing-20260213.113451-tentacle-distro-default-trial/50513
2026-02-14T01:05:03.137 INFO:teuthology.dispatcher.supervisor:Job PID: 1415553
2026-02-14T01:05:03.137 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-14T01:07:03.138 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:09:03.241 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:11:03.316 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:13:03.442 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:15:03.523 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:17:03.646 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:19:03.750 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:21:03.875 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:23:03.957 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:25:04.046 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:27:04.172 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:29:04.242 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:31:04.329 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:33:04.450 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:35:04.578 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:37:04.727 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:39:04.857 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:41:04.933 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:43:05.051 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:43:05.123 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2026-02-14T01:43:05.292 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-14T01:43:06.009 INFO:teuthology.orchestra.console.trial028:Power off
2026-02-14T01:43:06.009 DEBUG:teuthology.orchestra.console.trial028:pexpect command: ipmitool -H trial028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T01:43:06.022 INFO:teuthology.orchestra.console.trial153:Power off
2026-02-14T01:43:06.022 DEBUG:teuthology.orchestra.console.trial153:pexpect command: ipmitool -H trial153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T01:43:06.033 INFO:teuthology.orchestra.console.trial156:Power off
2026-02-14T01:43:06.033 DEBUG:teuthology.orchestra.console.trial156:pexpect command: ipmitool -H trial156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T01:43:06.050 DEBUG:teuthology.orchestra.console.trial028:power off output: Chassis Power Control: Down/Off
2026-02-14T01:43:06.061 DEBUG:teuthology.orchestra.console.trial153:power off output: Chassis Power Control: Down/Off
2026-02-14T01:43:06.072 DEBUG:teuthology.orchestra.console.trial156:power off output: Chassis Power Control: Down/Off
2026-02-14T01:43:10.051 DEBUG:teuthology.orchestra.console.trial028:pexpect command: ipmitool -H trial028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:43:10.061 DEBUG:teuthology.orchestra.console.trial153:pexpect command: ipmitool -H trial153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:43:10.073 DEBUG:teuthology.orchestra.console.trial156:pexpect command: ipmitool -H trial156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:43:10.162 DEBUG:teuthology.orchestra.console.trial028:check power output: Chassis Power is on
2026-02-14T01:43:10.172 DEBUG:teuthology.orchestra.console.trial153:check power output: Chassis Power is on
2026-02-14T01:43:10.183 DEBUG:teuthology.orchestra.console.trial156:check power output: Chassis Power is on
2026-02-14T01:43:14.163 DEBUG:teuthology.orchestra.console.trial028:pexpect command: ipmitool -H trial028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:43:14.173 DEBUG:teuthology.orchestra.console.trial153:pexpect command: ipmitool -H trial153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:43:14.184 DEBUG:teuthology.orchestra.console.trial156:pexpect command: ipmitool -H trial156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:43:14.272 DEBUG:teuthology.orchestra.console.trial028:check power output: Chassis Power is off
2026-02-14T01:43:14.272 INFO:teuthology.orchestra.console.trial028:Power off completed
2026-02-14T01:43:14.283 DEBUG:teuthology.orchestra.console.trial153:check power output: Chassis Power is off
2026-02-14T01:43:14.283 INFO:teuthology.orchestra.console.trial153:Power off completed
2026-02-14T01:43:14.292 DEBUG:teuthology.orchestra.console.trial156:check power output: Chassis Power is off
2026-02-14T01:43:14.292 INFO:teuthology.orchestra.console.trial156:Power off completed
2026-02-14T01:43:14.554 INFO:teuthology.lock.ops:unlocked: trial028.front.sepia.ceph.com
2026-02-14T01:43:14.563 INFO:teuthology.lock.ops:unlocked: trial153.front.sepia.ceph.com
2026-02-14T01:43:14.566 INFO:teuthology.lock.ops:unlocked: trial156.front.sepia.ceph.com