2026-02-14T01:36:33.867 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71
2026-02-14T01:36:33.867 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:36:33.939 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:36:34.197 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x71615e540b00>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x71615e4ab240>, signals=[15])
2026-02-14T01:36:34.197 INFO:teuthology.lock.ops:Start node 'trial053.front.sepia.ceph.com' reimaging
2026-02-14T01:36:34.197 INFO:teuthology.lock.ops:Updating [trial053.front.sepia.ceph.com]: reset os type and version on server
2026-02-14T01:36:34.197 INFO:teuthology.lock.ops:Updating trial053.front.sepia.ceph.com on lock server
2026-02-14T01:36:34.254 INFO:teuthology.lock.ops:Start node 'trial118.front.sepia.ceph.com' reimaging
2026-02-14T01:36:34.254 INFO:teuthology.lock.ops:Updating [trial118.front.sepia.ceph.com]: reset os type and version on server
2026-02-14T01:36:34.254 INFO:teuthology.lock.ops:Updating trial118.front.sepia.ceph.com on lock server
2026-02-14T01:36:34.311 INFO:teuthology.lock.ops:Start node 'trial184.front.sepia.ceph.com' reimaging
2026-02-14T01:36:34.311 INFO:teuthology.lock.ops:Updating [trial184.front.sepia.ceph.com]: reset os type and version on server
2026-02-14T01:36:34.311 INFO:teuthology.lock.ops:Updating trial184.front.sepia.ceph.com on lock server
2026-02-14T01:36:34.543 DEBUG:teuthology.provision.fog.trial118:Requesting image trial_centos_9.stream (ID 3)
2026-02-14T01:36:34.543 DEBUG:teuthology.provision.fog.trial053:Requesting image trial_centos_9.stream (ID 3)
2026-02-14T01:36:34.570 INFO:teuthology.provision.fog.trial053:Scheduling deploy of centos 9.stream
2026-02-14T01:36:34.586 INFO:teuthology.provision.fog.trial118:Scheduling deploy of centos 9.stream
2026-02-14T01:36:34.608 DEBUG:teuthology.provision.fog.trial184:Requesting image trial_centos_9.stream (ID 3)
2026-02-14T01:36:34.660 INFO:teuthology.provision.fog.trial184:Scheduling deploy of centos 9.stream
2026-02-14T01:36:35.667 INFO:teuthology.orchestra.console.trial053:Power off
2026-02-14T01:36:35.667 DEBUG:teuthology.orchestra.console.trial053:pexpect command: ipmitool -H trial053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T01:36:35.729 DEBUG:teuthology.orchestra.console.trial053:power off output: Chassis Power Control: Down/Off
2026-02-14T01:36:35.778 INFO:teuthology.orchestra.console.trial184:Power off
2026-02-14T01:36:35.778 DEBUG:teuthology.orchestra.console.trial184:pexpect command: ipmitool -H trial184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T01:36:35.795 INFO:teuthology.orchestra.console.trial118:Power off
2026-02-14T01:36:35.796 DEBUG:teuthology.orchestra.console.trial118:pexpect command: ipmitool -H trial118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T01:36:35.816 DEBUG:teuthology.orchestra.console.trial184:power off output: Chassis Power Control: Down/Off
2026-02-14T01:36:35.837 DEBUG:teuthology.orchestra.console.trial118:power off output: Chassis Power Control: Down/Off
2026-02-14T01:36:39.734 DEBUG:teuthology.orchestra.console.trial053:pexpect command: ipmitool -H trial053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:36:39.817 DEBUG:teuthology.orchestra.console.trial184:pexpect command: ipmitool -H trial184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:36:39.838 DEBUG:teuthology.orchestra.console.trial118:pexpect command: ipmitool -H trial118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:36:39.848 DEBUG:teuthology.orchestra.console.trial053:check power output: Chassis Power is off
2026-02-14T01:36:39.848 INFO:teuthology.orchestra.console.trial053:Power off completed
2026-02-14T01:36:39.924 DEBUG:teuthology.orchestra.console.trial184:check power output: Chassis Power is off
2026-02-14T01:36:39.925 INFO:teuthology.orchestra.console.trial184:Power off completed
2026-02-14T01:36:39.947 DEBUG:teuthology.orchestra.console.trial118:check power output: Chassis Power is off
2026-02-14T01:36:39.948 INFO:teuthology.orchestra.console.trial118:Power off completed
2026-02-14T01:36:39.949 INFO:teuthology.orchestra.console.trial053:Power on
2026-02-14T01:36:39.949 DEBUG:teuthology.orchestra.console.trial053:pexpect command: ipmitool -H trial053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-14T01:36:39.988 DEBUG:teuthology.orchestra.console.trial053:power on output: Chassis Power Control: Up/On
2026-02-14T01:36:40.025 INFO:teuthology.orchestra.console.trial184:Power on
2026-02-14T01:36:40.025 DEBUG:teuthology.orchestra.console.trial184:pexpect command: ipmitool -H trial184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-14T01:36:40.049 INFO:teuthology.orchestra.console.trial118:Power on
2026-02-14T01:36:40.049 DEBUG:teuthology.orchestra.console.trial118:pexpect command: ipmitool -H trial118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-14T01:36:40.064 DEBUG:teuthology.orchestra.console.trial184:power on output: Chassis Power Control: Up/On
2026-02-14T01:36:40.090 DEBUG:teuthology.orchestra.console.trial118:power on output: Chassis Power Control: Up/On
2026-02-14T01:36:43.993 DEBUG:teuthology.orchestra.console.trial053:pexpect command: ipmitool -H trial053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:36:44.066 DEBUG:teuthology.orchestra.console.trial184:pexpect command: ipmitool -H trial184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:36:44.091 DEBUG:teuthology.orchestra.console.trial118:pexpect command: ipmitool -H trial118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:36:44.103 DEBUG:teuthology.orchestra.console.trial053:check power output: Chassis Power is on
2026-02-14T01:36:44.104 INFO:teuthology.orchestra.console.trial053:Power on completed
2026-02-14T01:36:44.177 DEBUG:teuthology.orchestra.console.trial184:check power output: Chassis Power is on
2026-02-14T01:36:44.177 INFO:teuthology.orchestra.console.trial184:Power on completed
2026-02-14T01:36:44.202 DEBUG:teuthology.orchestra.console.trial118:check power output: Chassis Power is on
2026-02-14T01:36:44.202 INFO:teuthology.orchestra.console.trial118:Power on completed
2026-02-14T01:36:44.204 INFO:teuthology.provision.fog.trial053:Waiting for deploy to finish
2026-02-14T01:36:44.278 INFO:teuthology.provision.fog.trial184:Waiting for deploy to finish
2026-02-14T01:36:44.303 INFO:teuthology.provision.fog.trial118:Waiting for deploy to finish
2026-02-14T01:38:31.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:38:31.736 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:38:31.894 ERROR:teuthology.orchestra.connection:Error authenticating with trial053.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2026-02-14T01:38:46.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:38:50.013 WARNING:teuthology.provision.fog.trial184:[Errno None] Unable to connect to port 22 on 10.20.193.184
2026-02-14T01:38:56.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:38:59.101 WARNING:teuthology.provision.fog.trial184:[Errno None] Unable to connect to port 22 on 10.20.193.184
2026-02-14T01:39:05.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:39:08.189 WARNING:teuthology.provision.fog.trial184:[Errno None] Unable to connect to port 22 on 10.20.193.184
2026-02-14T01:39:14.190 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:39:17.277 WARNING:teuthology.provision.fog.trial184:[Errno None] Unable to connect to port 22 on 10.20.193.184
2026-02-14T01:39:23.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:39:31.332 WARNING:teuthology.provision.fog.trial118:timed out
2026-02-14T01:39:35.897 WARNING:teuthology.provision.fog.trial053:timed out
2026-02-14T01:39:37.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:39:41.899 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:39:42.623 ERROR:teuthology.orchestra.connection:Error authenticating with trial118.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-14T01:39:46.137 ERROR:teuthology.orchestra.connection:Error authenticating with trial053.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-14T01:39:46.707 ERROR:teuthology.orchestra.connection:Error authenticating with trial118.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-14T01:39:50.226 ERROR:teuthology.orchestra.connection:Error authenticating with trial053.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-14T01:39:53.821 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-14T01:39:53.821 DEBUG:teuthology.orchestra.run.trial118:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-14T01:39:53.987 INFO:teuthology.provision.fog.trial118:Node is ready
2026-02-14T01:39:53.987 DEBUG:teuthology.orchestra.run.trial118:> hostname
2026-02-14T01:39:54.044 INFO:teuthology.orchestra.run.trial118.stdout:trial118.front.sepia.ceph.com
2026-02-14T01:39:54.044 DEBUG:teuthology.orchestra.run.trial118:> grep trial118.front.sepia.ceph.com /etc/hosts
2026-02-14T01:39:54.101 INFO:teuthology.orchestra.run.trial118.stdout:10.20.193.118 trial118.front.sepia.ceph.com trial118
2026-02-14T01:39:54.102 DEBUG:teuthology.orchestra.run.trial118:> sudo hostname trial118
2026-02-14T01:39:54.172 DEBUG:teuthology.orchestra.run.trial118:> sudo sed -i -e 's/trial118.front.sepia.ceph.com/trial118/g' /etc/hosts
2026-02-14T01:39:54.237 DEBUG:teuthology.orchestra.run.trial118:> sudo sed -i -e 's/10.20.193.118/10.20.193.118/g' /etc/hosts
2026-02-14T01:39:54.302 DEBUG:teuthology.orchestra.run.trial118:> sudo sed -i -e 's/trial118.front.sepia.ceph.com/trial118/g' /etc/hostname
2026-02-14T01:39:54.367 DEBUG:teuthology.orchestra.run.trial118:> sudo hostname trial118
2026-02-14T01:39:54.431 DEBUG:teuthology.orchestra.run.trial118:> cat /etc/os-release
2026-02-14T01:39:54.488 INFO:teuthology.orchestra.run.trial118.stdout:NAME="CentOS Stream"
2026-02-14T01:39:54.488 INFO:teuthology.orchestra.run.trial118.stdout:VERSION="9"
2026-02-14T01:39:54.488 INFO:teuthology.orchestra.run.trial118.stdout:ID="centos"
2026-02-14T01:39:54.488 INFO:teuthology.orchestra.run.trial118.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:39:54.488 INFO:teuthology.orchestra.run.trial118.stdout:VERSION_ID="9"
2026-02-14T01:39:54.488 INFO:teuthology.orchestra.run.trial118.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:39:54.488 INFO:teuthology.orchestra.run.trial118.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:39:54.489 INFO:teuthology.orchestra.run.trial118.stdout:ANSI_COLOR="0;31"
2026-02-14T01:39:54.489 INFO:teuthology.orchestra.run.trial118.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:39:54.489 INFO:teuthology.orchestra.run.trial118.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:39:54.489 INFO:teuthology.orchestra.run.trial118.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:39:54.489 INFO:teuthology.orchestra.run.trial118.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:39:54.489 INFO:teuthology.orchestra.run.trial118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:39:54.489 INFO:teuthology.orchestra.run.trial118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:39:54.490 INFO:teuthology.provision.fog.trial118:Deploy complete!
2026-02-14T01:39:54.491 DEBUG:teuthology.parallel:result is None
2026-02-14T01:39:57.336 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-14T01:39:57.336 DEBUG:teuthology.orchestra.run.trial053:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-14T01:39:57.500 INFO:teuthology.provision.fog.trial053:Node is ready
2026-02-14T01:39:57.501 DEBUG:teuthology.orchestra.run.trial053:> hostname
2026-02-14T01:39:57.555 INFO:teuthology.orchestra.run.trial053.stdout:trial053.front.sepia.ceph.com
2026-02-14T01:39:57.555 DEBUG:teuthology.orchestra.run.trial053:> grep trial053.front.sepia.ceph.com /etc/hosts
2026-02-14T01:39:57.609 INFO:teuthology.orchestra.run.trial053.stdout:10.20.193.53 trial053.front.sepia.ceph.com trial053
2026-02-14T01:39:57.610 DEBUG:teuthology.orchestra.run.trial053:> sudo hostname trial053
2026-02-14T01:39:57.678 DEBUG:teuthology.orchestra.run.trial053:> sudo sed -i -e 's/trial053.front.sepia.ceph.com/trial053/g' /etc/hosts
2026-02-14T01:39:57.744 DEBUG:teuthology.orchestra.run.trial053:> sudo sed -i -e 's/10.20.193.53/10.20.193.53/g' /etc/hosts
2026-02-14T01:39:57.809 DEBUG:teuthology.orchestra.run.trial053:> sudo sed -i -e 's/trial053.front.sepia.ceph.com/trial053/g' /etc/hostname
2026-02-14T01:39:57.872 DEBUG:teuthology.orchestra.run.trial053:> sudo hostname trial053
2026-02-14T01:39:57.934 DEBUG:teuthology.orchestra.run.trial053:> cat /etc/os-release
2026-02-14T01:39:57.990 INFO:teuthology.orchestra.run.trial053.stdout:NAME="CentOS Stream"
2026-02-14T01:39:57.991 INFO:teuthology.orchestra.run.trial053.stdout:VERSION="9"
2026-02-14T01:39:57.991 INFO:teuthology.orchestra.run.trial053.stdout:ID="centos"
2026-02-14T01:39:57.991 INFO:teuthology.orchestra.run.trial053.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:39:57.991 INFO:teuthology.orchestra.run.trial053.stdout:VERSION_ID="9"
2026-02-14T01:39:57.991 INFO:teuthology.orchestra.run.trial053.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:39:57.991 INFO:teuthology.orchestra.run.trial053.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:39:57.991 INFO:teuthology.orchestra.run.trial053.stdout:ANSI_COLOR="0;31"
2026-02-14T01:39:57.992 INFO:teuthology.orchestra.run.trial053.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:39:57.992 INFO:teuthology.orchestra.run.trial053.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:39:57.992 INFO:teuthology.orchestra.run.trial053.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:39:57.992 INFO:teuthology.orchestra.run.trial053.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:39:57.992 INFO:teuthology.orchestra.run.trial053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:39:57.992 INFO:teuthology.orchestra.run.trial053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:39:57.992 INFO:teuthology.provision.fog.trial053:Deploy complete!
2026-02-14T01:39:57.993 DEBUG:teuthology.parallel:result is None
2026-02-14T01:39:59.366 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-14T01:39:59.366 DEBUG:teuthology.orchestra.run.trial184:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-14T01:39:59.522 INFO:teuthology.provision.fog.trial184:Node is ready
2026-02-14T01:39:59.522 DEBUG:teuthology.orchestra.run.trial184:> hostname
2026-02-14T01:39:59.579 INFO:teuthology.orchestra.run.trial184.stdout:trial184.front.sepia.ceph.com
2026-02-14T01:39:59.579 DEBUG:teuthology.orchestra.run.trial184:> grep trial184.front.sepia.ceph.com /etc/hosts
2026-02-14T01:39:59.636 INFO:teuthology.orchestra.run.trial184.stdout:10.20.193.184 trial184.front.sepia.ceph.com trial184
2026-02-14T01:39:59.637 DEBUG:teuthology.orchestra.run.trial184:> sudo hostname trial184
2026-02-14T01:39:59.705 DEBUG:teuthology.orchestra.run.trial184:> sudo sed -i -e 's/trial184.front.sepia.ceph.com/trial184/g' /etc/hosts
2026-02-14T01:39:59.770 DEBUG:teuthology.orchestra.run.trial184:> sudo sed -i -e 's/10.20.193.184/10.20.193.184/g' /etc/hosts
2026-02-14T01:39:59.835 DEBUG:teuthology.orchestra.run.trial184:> sudo sed -i -e 's/trial184.front.sepia.ceph.com/trial184/g' /etc/hostname
2026-02-14T01:39:59.899 DEBUG:teuthology.orchestra.run.trial184:> sudo hostname trial184
2026-02-14T01:39:59.963 DEBUG:teuthology.orchestra.run.trial184:> cat /etc/os-release
2026-02-14T01:40:00.020 INFO:teuthology.orchestra.run.trial184.stdout:NAME="CentOS Stream"
2026-02-14T01:40:00.021 INFO:teuthology.orchestra.run.trial184.stdout:VERSION="9"
2026-02-14T01:40:00.021 INFO:teuthology.orchestra.run.trial184.stdout:ID="centos"
2026-02-14T01:40:00.021 INFO:teuthology.orchestra.run.trial184.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:40:00.021 INFO:teuthology.orchestra.run.trial184.stdout:VERSION_ID="9"
2026-02-14T01:40:00.021 INFO:teuthology.orchestra.run.trial184.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:40:00.021 INFO:teuthology.orchestra.run.trial184.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:40:00.021 INFO:teuthology.orchestra.run.trial184.stdout:ANSI_COLOR="0;31"
2026-02-14T01:40:00.021 INFO:teuthology.orchestra.run.trial184.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:40:00.021 INFO:teuthology.orchestra.run.trial184.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:40:00.021 INFO:teuthology.orchestra.run.trial184.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:40:00.022 INFO:teuthology.orchestra.run.trial184.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:40:00.022 INFO:teuthology.orchestra.run.trial184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:40:00.022 INFO:teuthology.orchestra.run.trial184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:40:00.022 INFO:teuthology.provision.fog.trial184:Deploy complete!
2026-02-14T01:40:00.023 DEBUG:teuthology.parallel:result is None
2026-02-14T01:40:00.543 INFO:teuthology.lock.ops:Checking trial053.front.sepia.ceph.com
2026-02-14T01:40:00.543 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-14T01:40:00.602 INFO:teuthology.lock.ops:Checking trial118.front.sepia.ceph.com
2026-02-14T01:40:00.602 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-14T01:40:00.660 INFO:teuthology.lock.ops:Checking trial184.front.sepia.ceph.com
2026-02-14T01:40:00.660 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-14T01:40:00.718 INFO:teuthology.lock.ops:Updating [trial053.front.sepia.ceph.com]: set os type and version on server
2026-02-14T01:40:00.718 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial053.front.sepia.ceph.com'
2026-02-14T01:40:00.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:40:00.778 DEBUG:teuthology.orchestra.run.trial053:> true
2026-02-14T01:40:00.846 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial053.front.sepia.ceph.com'
2026-02-14T01:40:00.846 DEBUG:teuthology.orchestra.run.trial053:> uname -m
2026-02-14T01:40:00.902 INFO:teuthology.orchestra.run.trial053.stdout:x86_64
2026-02-14T01:40:00.902 DEBUG:teuthology.orchestra.run.trial053:> cat /etc/os-release
2026-02-14T01:40:00.958 INFO:teuthology.orchestra.run.trial053.stdout:NAME="CentOS Stream"
2026-02-14T01:40:00.959 INFO:teuthology.orchestra.run.trial053.stdout:VERSION="9"
2026-02-14T01:40:00.959 INFO:teuthology.orchestra.run.trial053.stdout:ID="centos"
2026-02-14T01:40:00.959 INFO:teuthology.orchestra.run.trial053.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:40:00.959 INFO:teuthology.orchestra.run.trial053.stdout:VERSION_ID="9"
2026-02-14T01:40:00.959 INFO:teuthology.orchestra.run.trial053.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:40:00.959 INFO:teuthology.orchestra.run.trial053.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:40:00.959 INFO:teuthology.orchestra.run.trial053.stdout:ANSI_COLOR="0;31"
2026-02-14T01:40:00.959 INFO:teuthology.orchestra.run.trial053.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:40:00.959 INFO:teuthology.orchestra.run.trial053.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:40:00.959 INFO:teuthology.orchestra.run.trial053.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:40:00.959 INFO:teuthology.orchestra.run.trial053.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:40:00.959 INFO:teuthology.orchestra.run.trial053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:40:00.959 INFO:teuthology.orchestra.run.trial053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:40:00.960 INFO:teuthology.lock.ops:Updating trial053.front.sepia.ceph.com on lock server
2026-02-14T01:40:01.043 INFO:teuthology.lock.ops:Updating [trial118.front.sepia.ceph.com]: set os type and version on server
2026-02-14T01:40:01.043 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial118.front.sepia.ceph.com'
2026-02-14T01:40:01.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:40:01.103 DEBUG:teuthology.orchestra.run.trial118:> true
2026-02-14T01:40:01.168 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial118.front.sepia.ceph.com'
2026-02-14T01:40:01.168 DEBUG:teuthology.orchestra.run.trial118:> uname -m
2026-02-14T01:40:01.224 INFO:teuthology.orchestra.run.trial118.stdout:x86_64
2026-02-14T01:40:01.224 DEBUG:teuthology.orchestra.run.trial118:> cat /etc/os-release
2026-02-14T01:40:01.281 INFO:teuthology.orchestra.run.trial118.stdout:NAME="CentOS Stream"
2026-02-14T01:40:01.281 INFO:teuthology.orchestra.run.trial118.stdout:VERSION="9"
2026-02-14T01:40:01.281 INFO:teuthology.orchestra.run.trial118.stdout:ID="centos"
2026-02-14T01:40:01.281 INFO:teuthology.orchestra.run.trial118.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:40:01.281 INFO:teuthology.orchestra.run.trial118.stdout:VERSION_ID="9"
2026-02-14T01:40:01.281 INFO:teuthology.orchestra.run.trial118.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:40:01.281 INFO:teuthology.orchestra.run.trial118.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:40:01.281 INFO:teuthology.orchestra.run.trial118.stdout:ANSI_COLOR="0;31"
2026-02-14T01:40:01.282 INFO:teuthology.orchestra.run.trial118.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:40:01.282 INFO:teuthology.orchestra.run.trial118.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:40:01.282 INFO:teuthology.orchestra.run.trial118.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:40:01.282 INFO:teuthology.orchestra.run.trial118.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:40:01.282 INFO:teuthology.orchestra.run.trial118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:40:01.282 INFO:teuthology.orchestra.run.trial118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:40:01.283 INFO:teuthology.lock.ops:Updating trial118.front.sepia.ceph.com on lock server
2026-02-14T01:40:01.381 INFO:teuthology.lock.ops:Updating [trial184.front.sepia.ceph.com]: set os type and version on server
2026-02-14T01:40:01.381 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial184.front.sepia.ceph.com'
2026-02-14T01:40:01.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:40:01.445 DEBUG:teuthology.orchestra.run.trial184:> true
2026-02-14T01:40:01.517 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial184.front.sepia.ceph.com'
2026-02-14T01:40:01.517 DEBUG:teuthology.orchestra.run.trial184:> uname -m
2026-02-14T01:40:01.573 INFO:teuthology.orchestra.run.trial184.stdout:x86_64
2026-02-14T01:40:01.573 DEBUG:teuthology.orchestra.run.trial184:> cat /etc/os-release
2026-02-14T01:40:01.630 INFO:teuthology.orchestra.run.trial184.stdout:NAME="CentOS Stream"
2026-02-14T01:40:01.630 INFO:teuthology.orchestra.run.trial184.stdout:VERSION="9"
2026-02-14T01:40:01.630 INFO:teuthology.orchestra.run.trial184.stdout:ID="centos"
2026-02-14T01:40:01.630 INFO:teuthology.orchestra.run.trial184.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:40:01.630 INFO:teuthology.orchestra.run.trial184.stdout:VERSION_ID="9"
2026-02-14T01:40:01.630 INFO:teuthology.orchestra.run.trial184.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:40:01.630 INFO:teuthology.orchestra.run.trial184.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:40:01.630 INFO:teuthology.orchestra.run.trial184.stdout:ANSI_COLOR="0;31"
2026-02-14T01:40:01.630 INFO:teuthology.orchestra.run.trial184.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:40:01.631 INFO:teuthology.orchestra.run.trial184.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:40:01.631 INFO:teuthology.orchestra.run.trial184.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:40:01.631 INFO:teuthology.orchestra.run.trial184.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:40:01.631 INFO:teuthology.orchestra.run.trial184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:40:01.631 INFO:teuthology.orchestra.run.trial184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:40:01.632 INFO:teuthology.lock.ops:Updating trial184.front.sepia.ceph.com on lock server
2026-02-14T01:40:01.738 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:40:01.871 INFO:teuthology.dispatcher.supervisor:Running job 50575
2026-02-14T01:40:01.871 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/bin/teuthology -v --owner scheduled_jcollin@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/jcollin-2026-02-14_00:12:18-fs-wip-jcollin-testing-20260213.113451-tentacle-distro-default-trial/50575 --name jcollin-2026-02-14_00:12:18-fs-wip-jcollin-testing-20260213.113451-tentacle-distro-default-trial --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/fsync-tester}} -- /home/teuthworker/mnt/teuthology/jcollin-2026-02-14_00:12:18-fs-wip-jcollin-testing-20260213.113451-tentacle-distro-default-trial/50575/orig.config.yaml
2026-02-14T01:40:01.872 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/jcollin-2026-02-14_00:12:18-fs-wip-jcollin-testing-20260213.113451-tentacle-distro-default-trial/50575
2026-02-14T01:40:01.872 INFO:teuthology.dispatcher.supervisor:Job PID: 1605726
2026-02-14T01:40:01.872 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-14T01:42:01.873 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:44:01.994 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:46:02.114 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:48:02.236 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:50:02.358 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:52:02.472 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:54:02.571 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:56:02.654 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:58:02.777 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T02:00:02.873 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T02:02:02.996 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T02:04:03.119 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T02:06:03.243 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T02:08:03.374 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T02:10:03.494 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T02:10:03.596 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-14T02:10:03.776 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-14T02:10:04.507 INFO:teuthology.orchestra.console.trial118:Power off
2026-02-14T02:10:04.507 DEBUG:teuthology.orchestra.console.trial118:pexpect command: ipmitool -H trial118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T02:10:04.519 INFO:teuthology.orchestra.console.trial184:Power off
2026-02-14T02:10:04.519 DEBUG:teuthology.orchestra.console.trial184:pexpect command: ipmitool -H trial184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T02:10:04.530 INFO:teuthology.orchestra.console.trial053:Power off
2026-02-14T02:10:04.531 DEBUG:teuthology.orchestra.console.trial053:pexpect command: ipmitool -H trial053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T02:10:04.549 DEBUG:teuthology.orchestra.console.trial118:power off output: Chassis Power Control: Down/Off
2026-02-14T02:10:04.560 DEBUG:teuthology.orchestra.console.trial184:power off output: Chassis Power Control: Down/Off
2026-02-14T02:10:04.568 DEBUG:teuthology.orchestra.console.trial053:power off output: Chassis Power Control: Down/Off
2026-02-14T02:10:08.550 DEBUG:teuthology.orchestra.console.trial118:pexpect command: ipmitool -H trial118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T02:10:08.561 DEBUG:teuthology.orchestra.console.trial184:pexpect command: ipmitool -H trial184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T02:10:08.569 DEBUG:teuthology.orchestra.console.trial053:pexpect command: ipmitool -H trial053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T02:10:08.657 DEBUG:teuthology.orchestra.console.trial118:check power output: Chassis Power is on
2026-02-14T02:10:08.670 DEBUG:teuthology.orchestra.console.trial184:check power output: Chassis Power is on
2026-02-14T02:10:08.678 DEBUG:teuthology.orchestra.console.trial053:check power output: Chassis Power is on
2026-02-14T02:10:12.658 DEBUG:teuthology.orchestra.console.trial118:pexpect command: ipmitool -H trial118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T02:10:12.671 DEBUG:teuthology.orchestra.console.trial184:pexpect command: ipmitool -H trial184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T02:10:12.678 DEBUG:teuthology.orchestra.console.trial053:pexpect command: ipmitool -H trial053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T02:10:12.768 DEBUG:teuthology.orchestra.console.trial118:check power output: Chassis Power is off
2026-02-14T02:10:12.768 INFO:teuthology.orchestra.console.trial118:Power off completed
2026-02-14T02:10:12.780 DEBUG:teuthology.orchestra.console.trial184:check power output: Chassis Power is off
2026-02-14T02:10:12.780 INFO:teuthology.orchestra.console.trial184:Power off completed
2026-02-14T02:10:12.787 DEBUG:teuthology.orchestra.console.trial053:check power output: Chassis Power is off
2026-02-14T02:10:12.787 INFO:teuthology.orchestra.console.trial053:Power off completed
2026-02-14T02:10:13.071 INFO:teuthology.lock.ops:unlocked: trial118.front.sepia.ceph.com
2026-02-14T02:10:13.080 INFO:teuthology.lock.ops:unlocked: trial184.front.sepia.ceph.com
2026-02-14T02:10:13.084 INFO:teuthology.lock.ops:unlocked: trial053.front.sepia.ceph.com