2026-02-22T06:07:24.826 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-22T06:07:24.827 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-22T06:07:24.911 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-22T06:07:25.165 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x780805c556d0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x78080546ed40>, signals=[15])
2026-02-22T06:07:25.165 INFO:teuthology.lock.ops:Start node 'trial121.front.sepia.ceph.com' reimaging
2026-02-22T06:07:25.166 INFO:teuthology.lock.ops:Updating [trial121.front.sepia.ceph.com]: reset os type and version on server
2026-02-22T06:07:25.166 INFO:teuthology.lock.ops:Updating trial121.front.sepia.ceph.com on lock server
2026-02-22T06:07:25.222 INFO:teuthology.lock.ops:Start node 'trial127.front.sepia.ceph.com' reimaging
2026-02-22T06:07:25.222 INFO:teuthology.lock.ops:Updating [trial127.front.sepia.ceph.com]: reset os type and version on server
2026-02-22T06:07:25.223 INFO:teuthology.lock.ops:Updating trial127.front.sepia.ceph.com on lock server
2026-02-22T06:07:25.280 INFO:teuthology.lock.ops:Start node 'trial177.front.sepia.ceph.com' reimaging
2026-02-22T06:07:25.280 INFO:teuthology.lock.ops:Updating [trial177.front.sepia.ceph.com]: reset os type and version on server
2026-02-22T06:07:25.280 INFO:teuthology.lock.ops:Updating trial177.front.sepia.ceph.com on lock server
2026-02-22T06:07:25.505 DEBUG:teuthology.provision.fog.trial121:Requesting image trial_centos_9.stream (ID 3)
2026-02-22T06:07:25.505 DEBUG:teuthology.provision.fog.trial127:Requesting image trial_centos_9.stream (ID 3)
2026-02-22T06:07:25.541 DEBUG:teuthology.provision.fog.trial177:Requesting image trial_centos_9.stream (ID 3)
2026-02-22T06:07:25.548 INFO:teuthology.provision.fog.trial127:Scheduling deploy of centos 9.stream
2026-02-22T06:07:25.553 INFO:teuthology.provision.fog.trial121:Scheduling deploy of centos 9.stream
2026-02-22T06:07:25.590 INFO:teuthology.provision.fog.trial177:Scheduling deploy of centos 9.stream
2026-02-22T06:07:27.023 INFO:teuthology.orchestra.console.trial127:Power off
2026-02-22T06:07:27.023 DEBUG:teuthology.orchestra.console.trial127:pexpect command: ipmitool -H trial127.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-22T06:07:27.043 INFO:teuthology.orchestra.console.trial121:Power off
2026-02-22T06:07:27.043 DEBUG:teuthology.orchestra.console.trial121:pexpect command: ipmitool -H trial121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-22T06:07:27.052 INFO:teuthology.orchestra.console.trial177:Power off
2026-02-22T06:07:27.052 DEBUG:teuthology.orchestra.console.trial177:pexpect command: ipmitool -H trial177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-22T06:07:27.061 DEBUG:teuthology.orchestra.console.trial127:power off output: Chassis Power Control: Down/Off
2026-02-22T06:07:27.080 DEBUG:teuthology.orchestra.console.trial121:power off output: Chassis Power Control: Down/Off
2026-02-22T06:07:27.087 DEBUG:teuthology.orchestra.console.trial177:power off output: Chassis Power Control: Down/Off
2026-02-22T06:07:31.065 DEBUG:teuthology.orchestra.console.trial127:pexpect command: ipmitool -H trial127.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-22T06:07:31.081 DEBUG:teuthology.orchestra.console.trial121:pexpect command: ipmitool -H trial121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-22T06:07:31.088 DEBUG:teuthology.orchestra.console.trial177:pexpect command: ipmitool -H trial177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-22T06:07:31.175 DEBUG:teuthology.orchestra.console.trial127:check power output: Chassis Power is off
2026-02-22T06:07:31.175 INFO:teuthology.orchestra.console.trial127:Power off completed
2026-02-22T06:07:31.190 DEBUG:teuthology.orchestra.console.trial121:check power output: Chassis Power is off
2026-02-22T06:07:31.190 INFO:teuthology.orchestra.console.trial121:Power off completed
2026-02-22T06:07:31.196 DEBUG:teuthology.orchestra.console.trial177:check power output: Chassis Power is off
2026-02-22T06:07:31.196 INFO:teuthology.orchestra.console.trial177:Power off completed
2026-02-22T06:07:31.276 INFO:teuthology.orchestra.console.trial127:Power on
2026-02-22T06:07:31.277 DEBUG:teuthology.orchestra.console.trial127:pexpect command: ipmitool -H trial127.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-22T06:07:31.291 INFO:teuthology.orchestra.console.trial121:Power on
2026-02-22T06:07:31.292 DEBUG:teuthology.orchestra.console.trial121:pexpect command: ipmitool -H trial121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-22T06:07:31.300 INFO:teuthology.orchestra.console.trial177:Power on
2026-02-22T06:07:31.301 DEBUG:teuthology.orchestra.console.trial177:pexpect command: ipmitool -H trial177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-22T06:07:31.316 DEBUG:teuthology.orchestra.console.trial127:power on output: Chassis Power Control: Up/On
2026-02-22T06:07:31.330 DEBUG:teuthology.orchestra.console.trial121:power on output: Chassis Power Control: Up/On
2026-02-22T06:07:31.339 DEBUG:teuthology.orchestra.console.trial177:power on output: Chassis Power Control: Up/On
2026-02-22T06:07:35.320 DEBUG:teuthology.orchestra.console.trial127:pexpect command: ipmitool -H trial127.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-22T06:07:35.331 DEBUG:teuthology.orchestra.console.trial121:pexpect command: ipmitool -H trial121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-22T06:07:35.341 DEBUG:teuthology.orchestra.console.trial177:pexpect command: ipmitool -H trial177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-22T06:07:35.432 DEBUG:teuthology.orchestra.console.trial127:check power output: Chassis Power is on
2026-02-22T06:07:35.432 INFO:teuthology.orchestra.console.trial127:Power on completed
2026-02-22T06:07:35.441 DEBUG:teuthology.orchestra.console.trial121:check power output: Chassis Power is on
2026-02-22T06:07:35.441 INFO:teuthology.orchestra.console.trial121:Power on completed
2026-02-22T06:07:35.450 DEBUG:teuthology.orchestra.console.trial177:check power output: Chassis Power is on
2026-02-22T06:07:35.450 INFO:teuthology.orchestra.console.trial177:Power on completed
2026-02-22T06:07:35.534 INFO:teuthology.provision.fog.trial127:Waiting for deploy to finish
2026-02-22T06:07:35.542 INFO:teuthology.provision.fog.trial121:Waiting for deploy to finish
2026-02-22T06:07:35.551 INFO:teuthology.provision.fog.trial177:Waiting for deploy to finish
2026-02-22T06:09:23.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-22T06:09:23.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial127.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-22T06:09:39.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-22T06:09:42.080 WARNING:teuthology.provision.fog.trial121:[Errno None] Unable to connect to port 22 on 10.20.193.121
2026-02-22T06:09:48.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-22T06:09:51.168 WARNING:teuthology.provision.fog.trial121:[Errno None] Unable to connect to port 22 on 10.20.193.121
2026-02-22T06:09:57.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-22T06:10:00.255 WARNING:teuthology.provision.fog.trial121:[Errno None] Unable to connect to port 22 on 10.20.193.121
2026-02-22T06:10:06.260 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-22T06:10:09.344 WARNING:teuthology.provision.fog.trial121:[Errno None] Unable to connect to port 22 on 10.20.193.121
2026-02-22T06:10:15.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-22T06:10:23.793 WARNING:teuthology.provision.fog.trial177:timed out
2026-02-22T06:10:23.798 WARNING:teuthology.provision.fog.trial127:timed out
2026-02-22T06:10:29.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-22T06:10:29.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial127.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-22T06:10:37.170 ERROR:teuthology.orchestra.connection:Error authenticating with trial177.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-22T06:10:37.308 ERROR:teuthology.orchestra.connection:Error authenticating with trial127.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-22T06:10:41.281 ERROR:teuthology.orchestra.connection:Error authenticating with trial177.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-22T06:10:41.412 ERROR:teuthology.orchestra.connection:Error authenticating with trial127.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-22T06:10:48.374 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-22T06:10:48.375 DEBUG:teuthology.orchestra.run.trial177:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-22T06:10:48.497 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-22T06:10:48.497 DEBUG:teuthology.orchestra.run.trial127:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-22T06:10:48.547 INFO:teuthology.provision.fog.trial177:Node is ready
2026-02-22T06:10:48.547 DEBUG:teuthology.orchestra.run.trial177:> hostname
2026-02-22T06:10:48.603 INFO:teuthology.orchestra.run.trial177.stdout:trial177.front.sepia.ceph.com
2026-02-22T06:10:48.603 DEBUG:teuthology.orchestra.run.trial177:> grep trial177.front.sepia.ceph.com /etc/hosts
2026-02-22T06:10:48.657 INFO:teuthology.provision.fog.trial127:Node is ready
2026-02-22T06:10:48.658 DEBUG:teuthology.orchestra.run.trial127:> hostname
2026-02-22T06:10:48.660 INFO:teuthology.orchestra.run.trial177.stdout:10.20.193.177 trial177.front.sepia.ceph.com trial177
2026-02-22T06:10:48.660 DEBUG:teuthology.orchestra.run.trial177:> sudo hostname trial177
2026-02-22T06:10:48.713 INFO:teuthology.orchestra.run.trial127.stdout:trial127.front.sepia.ceph.com
2026-02-22T06:10:48.713 DEBUG:teuthology.orchestra.run.trial127:> grep trial127.front.sepia.ceph.com /etc/hosts
2026-02-22T06:10:48.729 DEBUG:teuthology.orchestra.run.trial177:> sudo sed -i -e 's/trial177.front.sepia.ceph.com/trial177/g' /etc/hosts
2026-02-22T06:10:48.768 INFO:teuthology.orchestra.run.trial127.stdout:10.20.193.127 trial127.front.sepia.ceph.com trial127
2026-02-22T06:10:48.768 DEBUG:teuthology.orchestra.run.trial127:> sudo hostname trial127
2026-02-22T06:10:48.793 DEBUG:teuthology.orchestra.run.trial177:> sudo sed -i -e 's/10.20.193.177/10.20.193.177/g' /etc/hosts
2026-02-22T06:10:48.838 DEBUG:teuthology.orchestra.run.trial127:> sudo sed -i -e 's/trial127.front.sepia.ceph.com/trial127/g' /etc/hosts
2026-02-22T06:10:48.858 DEBUG:teuthology.orchestra.run.trial177:> sudo sed -i -e 's/trial177.front.sepia.ceph.com/trial177/g' /etc/hostname
2026-02-22T06:10:48.903 DEBUG:teuthology.orchestra.run.trial127:> sudo sed -i -e 's/10.20.193.127/10.20.193.127/g' /etc/hosts
2026-02-22T06:10:48.921 DEBUG:teuthology.orchestra.run.trial177:> sudo hostname trial177
2026-02-22T06:10:48.967 DEBUG:teuthology.orchestra.run.trial127:> sudo sed -i -e 's/trial127.front.sepia.ceph.com/trial127/g' /etc/hostname
2026-02-22T06:10:48.985 DEBUG:teuthology.orchestra.run.trial177:> cat /etc/os-release
2026-02-22T06:10:49.031 DEBUG:teuthology.orchestra.run.trial127:> sudo hostname trial127
2026-02-22T06:10:49.040 INFO:teuthology.orchestra.run.trial177.stdout:NAME="CentOS Stream"
2026-02-22T06:10:49.041 INFO:teuthology.orchestra.run.trial177.stdout:VERSION="9"
2026-02-22T06:10:49.041 INFO:teuthology.orchestra.run.trial177.stdout:ID="centos"
2026-02-22T06:10:49.041 INFO:teuthology.orchestra.run.trial177.stdout:ID_LIKE="rhel fedora"
2026-02-22T06:10:49.041 INFO:teuthology.orchestra.run.trial177.stdout:VERSION_ID="9"
2026-02-22T06:10:49.041 INFO:teuthology.orchestra.run.trial177.stdout:PLATFORM_ID="platform:el9"
2026-02-22T06:10:49.041 INFO:teuthology.orchestra.run.trial177.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-22T06:10:49.041 INFO:teuthology.orchestra.run.trial177.stdout:ANSI_COLOR="0;31"
2026-02-22T06:10:49.041 INFO:teuthology.orchestra.run.trial177.stdout:LOGO="fedora-logo-icon"
2026-02-22T06:10:49.041 INFO:teuthology.orchestra.run.trial177.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-22T06:10:49.041 INFO:teuthology.orchestra.run.trial177.stdout:HOME_URL="https://centos.org/"
2026-02-22T06:10:49.041 INFO:teuthology.orchestra.run.trial177.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-22T06:10:49.041 INFO:teuthology.orchestra.run.trial177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-22T06:10:49.041 INFO:teuthology.orchestra.run.trial177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-22T06:10:49.041 INFO:teuthology.provision.fog.trial177:Deploy complete!
2026-02-22T06:10:49.042 DEBUG:teuthology.parallel:result is None
2026-02-22T06:10:49.094 DEBUG:teuthology.orchestra.run.trial127:> cat /etc/os-release
2026-02-22T06:10:49.149 INFO:teuthology.orchestra.run.trial127.stdout:NAME="CentOS Stream"
2026-02-22T06:10:49.149 INFO:teuthology.orchestra.run.trial127.stdout:VERSION="9"
2026-02-22T06:10:49.149 INFO:teuthology.orchestra.run.trial127.stdout:ID="centos"
2026-02-22T06:10:49.149 INFO:teuthology.orchestra.run.trial127.stdout:ID_LIKE="rhel fedora"
2026-02-22T06:10:49.149 INFO:teuthology.orchestra.run.trial127.stdout:VERSION_ID="9"
2026-02-22T06:10:49.149 INFO:teuthology.orchestra.run.trial127.stdout:PLATFORM_ID="platform:el9"
2026-02-22T06:10:49.149 INFO:teuthology.orchestra.run.trial127.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-22T06:10:49.149 INFO:teuthology.orchestra.run.trial127.stdout:ANSI_COLOR="0;31"
2026-02-22T06:10:49.149 INFO:teuthology.orchestra.run.trial127.stdout:LOGO="fedora-logo-icon"
2026-02-22T06:10:49.149 INFO:teuthology.orchestra.run.trial127.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-22T06:10:49.149 INFO:teuthology.orchestra.run.trial127.stdout:HOME_URL="https://centos.org/"
2026-02-22T06:10:49.149 INFO:teuthology.orchestra.run.trial127.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-22T06:10:49.149 INFO:teuthology.orchestra.run.trial127.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-22T06:10:49.149 INFO:teuthology.orchestra.run.trial127.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-22T06:10:49.149 INFO:teuthology.provision.fog.trial127:Deploy complete!
2026-02-22T06:10:49.150 DEBUG:teuthology.parallel:result is None
2026-02-22T06:10:51.523 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-22T06:10:51.523 DEBUG:teuthology.orchestra.run.trial121:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-22T06:10:51.739 INFO:teuthology.provision.fog.trial121:Node is ready
2026-02-22T06:10:51.739 DEBUG:teuthology.orchestra.run.trial121:> hostname
2026-02-22T06:10:51.799 INFO:teuthology.orchestra.run.trial121.stdout:trial121.front.sepia.ceph.com
2026-02-22T06:10:51.799 DEBUG:teuthology.orchestra.run.trial121:> grep trial121.front.sepia.ceph.com /etc/hosts
2026-02-22T06:10:51.859 INFO:teuthology.orchestra.run.trial121.stdout:10.20.193.121 trial121.front.sepia.ceph.com trial121
2026-02-22T06:10:51.860 DEBUG:teuthology.orchestra.run.trial121:> sudo hostname trial121
2026-02-22T06:10:51.933 DEBUG:teuthology.orchestra.run.trial121:> sudo sed -i -e 's/trial121.front.sepia.ceph.com/trial121/g' /etc/hosts
2026-02-22T06:10:52.002 DEBUG:teuthology.orchestra.run.trial121:> sudo sed -i -e 's/10.20.193.121/10.20.193.121/g' /etc/hosts
2026-02-22T06:10:52.071 DEBUG:teuthology.orchestra.run.trial121:> sudo sed -i -e 's/trial121.front.sepia.ceph.com/trial121/g' /etc/hostname
2026-02-22T06:10:52.138 DEBUG:teuthology.orchestra.run.trial121:> sudo hostname trial121
2026-02-22T06:10:52.202 DEBUG:teuthology.orchestra.run.trial121:> cat /etc/os-release
2026-02-22T06:10:52.258 INFO:teuthology.orchestra.run.trial121.stdout:NAME="CentOS Stream"
2026-02-22T06:10:52.258 INFO:teuthology.orchestra.run.trial121.stdout:VERSION="9"
2026-02-22T06:10:52.258 INFO:teuthology.orchestra.run.trial121.stdout:ID="centos"
2026-02-22T06:10:52.258 INFO:teuthology.orchestra.run.trial121.stdout:ID_LIKE="rhel fedora"
2026-02-22T06:10:52.258 INFO:teuthology.orchestra.run.trial121.stdout:VERSION_ID="9"
2026-02-22T06:10:52.258 INFO:teuthology.orchestra.run.trial121.stdout:PLATFORM_ID="platform:el9"
2026-02-22T06:10:52.258 INFO:teuthology.orchestra.run.trial121.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-22T06:10:52.258 INFO:teuthology.orchestra.run.trial121.stdout:ANSI_COLOR="0;31"
2026-02-22T06:10:52.258 INFO:teuthology.orchestra.run.trial121.stdout:LOGO="fedora-logo-icon"
2026-02-22T06:10:52.258 INFO:teuthology.orchestra.run.trial121.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-22T06:10:52.258 INFO:teuthology.orchestra.run.trial121.stdout:HOME_URL="https://centos.org/"
2026-02-22T06:10:52.258 INFO:teuthology.orchestra.run.trial121.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-22T06:10:52.258 INFO:teuthology.orchestra.run.trial121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-22T06:10:52.258 INFO:teuthology.orchestra.run.trial121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-22T06:10:52.259 INFO:teuthology.provision.fog.trial121:Deploy complete!
2026-02-22T06:10:52.259 DEBUG:teuthology.parallel:result is None
2026-02-22T06:10:52.765 INFO:teuthology.lock.ops:Checking trial121.front.sepia.ceph.com
2026-02-22T06:10:52.766 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-22T06:10:52.892 INFO:teuthology.lock.ops:Checking trial127.front.sepia.ceph.com
2026-02-22T06:10:52.892 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-22T06:10:52.958 INFO:teuthology.lock.ops:Checking trial177.front.sepia.ceph.com
2026-02-22T06:10:52.958 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-22T06:10:53.023 INFO:teuthology.lock.ops:Updating [trial121.front.sepia.ceph.com]: set os type and version on server
2026-02-22T06:10:53.023 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial121.front.sepia.ceph.com'
2026-02-22T06:10:53.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-22T06:10:53.085 DEBUG:teuthology.orchestra.run.trial121:> true
2026-02-22T06:10:53.158 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial121.front.sepia.ceph.com'
2026-02-22T06:10:53.158 DEBUG:teuthology.orchestra.run.trial121:> uname -m
2026-02-22T06:10:53.215 INFO:teuthology.orchestra.run.trial121.stdout:x86_64
2026-02-22T06:10:53.215 DEBUG:teuthology.orchestra.run.trial121:> cat /etc/os-release
2026-02-22T06:10:53.271 INFO:teuthology.orchestra.run.trial121.stdout:NAME="CentOS Stream"
2026-02-22T06:10:53.271 INFO:teuthology.orchestra.run.trial121.stdout:VERSION="9"
2026-02-22T06:10:53.271 INFO:teuthology.orchestra.run.trial121.stdout:ID="centos"
2026-02-22T06:10:53.271 INFO:teuthology.orchestra.run.trial121.stdout:ID_LIKE="rhel fedora"
2026-02-22T06:10:53.271 INFO:teuthology.orchestra.run.trial121.stdout:VERSION_ID="9"
2026-02-22T06:10:53.271 INFO:teuthology.orchestra.run.trial121.stdout:PLATFORM_ID="platform:el9"
2026-02-22T06:10:53.271 INFO:teuthology.orchestra.run.trial121.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-22T06:10:53.271 INFO:teuthology.orchestra.run.trial121.stdout:ANSI_COLOR="0;31"
2026-02-22T06:10:53.271 INFO:teuthology.orchestra.run.trial121.stdout:LOGO="fedora-logo-icon"
2026-02-22T06:10:53.271 INFO:teuthology.orchestra.run.trial121.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-22T06:10:53.271 INFO:teuthology.orchestra.run.trial121.stdout:HOME_URL="https://centos.org/"
2026-02-22T06:10:53.271 INFO:teuthology.orchestra.run.trial121.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-22T06:10:53.271 INFO:teuthology.orchestra.run.trial121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-22T06:10:53.271 INFO:teuthology.orchestra.run.trial121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-22T06:10:53.272 INFO:teuthology.lock.ops:Updating trial121.front.sepia.ceph.com on lock server
2026-02-22T06:10:53.334 INFO:teuthology.lock.ops:Updating [trial127.front.sepia.ceph.com]: set os type and version on server
2026-02-22T06:10:53.334 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial127.front.sepia.ceph.com'
2026-02-22T06:10:53.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial127.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-22T06:10:53.412 DEBUG:teuthology.orchestra.run.trial127:> true
2026-02-22T06:10:53.484 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial127.front.sepia.ceph.com'
2026-02-22T06:10:53.485 DEBUG:teuthology.orchestra.run.trial127:> uname -m
2026-02-22T06:10:53.540 INFO:teuthology.orchestra.run.trial127.stdout:x86_64
2026-02-22T06:10:53.540 DEBUG:teuthology.orchestra.run.trial127:> cat /etc/os-release
2026-02-22T06:10:53.597 INFO:teuthology.orchestra.run.trial127.stdout:NAME="CentOS Stream"
2026-02-22T06:10:53.597 INFO:teuthology.orchestra.run.trial127.stdout:VERSION="9"
2026-02-22T06:10:53.598 INFO:teuthology.orchestra.run.trial127.stdout:ID="centos"
2026-02-22T06:10:53.598 INFO:teuthology.orchestra.run.trial127.stdout:ID_LIKE="rhel fedora"
2026-02-22T06:10:53.598 INFO:teuthology.orchestra.run.trial127.stdout:VERSION_ID="9"
2026-02-22T06:10:53.598 INFO:teuthology.orchestra.run.trial127.stdout:PLATFORM_ID="platform:el9"
2026-02-22T06:10:53.598 INFO:teuthology.orchestra.run.trial127.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-22T06:10:53.598 INFO:teuthology.orchestra.run.trial127.stdout:ANSI_COLOR="0;31"
2026-02-22T06:10:53.599 INFO:teuthology.orchestra.run.trial127.stdout:LOGO="fedora-logo-icon"
2026-02-22T06:10:53.599 INFO:teuthology.orchestra.run.trial127.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-22T06:10:53.599 INFO:teuthology.orchestra.run.trial127.stdout:HOME_URL="https://centos.org/"
2026-02-22T06:10:53.599 INFO:teuthology.orchestra.run.trial127.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-22T06:10:53.599 INFO:teuthology.orchestra.run.trial127.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-22T06:10:53.599 INFO:teuthology.orchestra.run.trial127.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-22T06:10:53.600 INFO:teuthology.lock.ops:Updating trial127.front.sepia.ceph.com on lock server
2026-02-22T06:10:53.709 INFO:teuthology.lock.ops:Updating [trial177.front.sepia.ceph.com]: set os type and version on server
2026-02-22T06:10:53.709 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial177.front.sepia.ceph.com'
2026-02-22T06:10:53.710 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-22T06:10:53.775 DEBUG:teuthology.orchestra.run.trial177:> true
2026-02-22T06:10:53.844 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial177.front.sepia.ceph.com'
2026-02-22T06:10:53.845 DEBUG:teuthology.orchestra.run.trial177:> uname -m
2026-02-22T06:10:53.900 INFO:teuthology.orchestra.run.trial177.stdout:x86_64
2026-02-22T06:10:53.900 DEBUG:teuthology.orchestra.run.trial177:> cat /etc/os-release
2026-02-22T06:10:53.955 INFO:teuthology.orchestra.run.trial177.stdout:NAME="CentOS Stream"
2026-02-22T06:10:53.956 INFO:teuthology.orchestra.run.trial177.stdout:VERSION="9"
2026-02-22T06:10:53.956 INFO:teuthology.orchestra.run.trial177.stdout:ID="centos"
2026-02-22T06:10:53.956 INFO:teuthology.orchestra.run.trial177.stdout:ID_LIKE="rhel fedora"
2026-02-22T06:10:53.956 INFO:teuthology.orchestra.run.trial177.stdout:VERSION_ID="9"
2026-02-22T06:10:53.956 INFO:teuthology.orchestra.run.trial177.stdout:PLATFORM_ID="platform:el9"
2026-02-22T06:10:53.956 INFO:teuthology.orchestra.run.trial177.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-22T06:10:53.956 INFO:teuthology.orchestra.run.trial177.stdout:ANSI_COLOR="0;31"
2026-02-22T06:10:53.956 INFO:teuthology.orchestra.run.trial177.stdout:LOGO="fedora-logo-icon"
2026-02-22T06:10:53.956 INFO:teuthology.orchestra.run.trial177.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-22T06:10:53.956 INFO:teuthology.orchestra.run.trial177.stdout:HOME_URL="https://centos.org/"
2026-02-22T06:10:53.956 INFO:teuthology.orchestra.run.trial177.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-22T06:10:53.956 INFO:teuthology.orchestra.run.trial177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-22T06:10:53.956 INFO:teuthology.orchestra.run.trial177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-22T06:10:53.956 INFO:teuthology.lock.ops:Updating trial177.front.sepia.ceph.com on lock server
2026-02-22T06:10:54.025 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-22T06:10:54.117 INFO:teuthology.dispatcher.supervisor:Running job 63534
2026-02-22T06:10:54.117 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/bin/teuthology -v --owner scheduled_skanta@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/skanta-2026-02-22_05:22:34-rados-wip-bharath9-testing-2026-02-21-2044-tentacle-distro-default-trial/63534 --name skanta-2026-02-22_05:22:34-rados-wip-bharath9-testing-2026-02-21-2044-tentacle-distro-default-trial --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds} ec_optimizations/ec_optimizations_off mon_election/connectivity msgr-failures/osd-delay objectstore/{bluestore/{alloc$/{hybrid} base mem$/{normal-1} onode-segment$/{none} write$/{v1/{compr$/{no$/{no}} v1}}}} rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/mnt/teuthology/skanta-2026-02-22_05:22:34-rados-wip-bharath9-testing-2026-02-21-2044-tentacle-distro-default-trial/63534/orig.config.yaml
2026-02-22T06:10:54.117 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/skanta-2026-02-22_05:22:34-rados-wip-bharath9-testing-2026-02-21-2044-tentacle-distro-default-trial/63534
2026-02-22T06:10:54.117 INFO:teuthology.dispatcher.supervisor:Job PID: 2320500
2026-02-22T06:10:54.117 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-22T06:12:54.164 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-22T06:14:54.267 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-22T06:16:54.447 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-22T06:18:54.626 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-22T06:20:54.807 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-22T06:22:54.999 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-22T06:24:55.168 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-22T06:26:55.327 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-22T06:28:55.497 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-22T06:30:55.691 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-22T06:30:55.766 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-22T06:30:59.291 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-22T06:31:00.020 INFO:teuthology.orchestra.console.trial127:Power off
2026-02-22T06:31:00.020 DEBUG:teuthology.orchestra.console.trial127:pexpect command: ipmitool -H trial127.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-22T06:31:00.028 INFO:teuthology.orchestra.console.trial177:Power off
2026-02-22T06:31:00.028 DEBUG:teuthology.orchestra.console.trial177:pexpect command: ipmitool -H trial177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-22T06:31:00.048 INFO:teuthology.orchestra.console.trial121:Power off
2026-02-22T06:31:00.048 DEBUG:teuthology.orchestra.console.trial121:pexpect command: ipmitool -H trial121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-22T06:31:00.059 DEBUG:teuthology.orchestra.console.trial127:power off output: Chassis Power Control: Down/Off
2026-02-22T06:31:00.066 DEBUG:teuthology.orchestra.console.trial177:power off output: Chassis Power Control: Down/Off
2026-02-22T06:31:00.086 DEBUG:teuthology.orchestra.console.trial121:power off output: Chassis Power Control: Down/Off
2026-02-22T06:31:04.064 DEBUG:teuthology.orchestra.console.trial127:pexpect command: ipmitool -H trial127.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-22T06:31:04.071 DEBUG:teuthology.orchestra.console.trial177:pexpect command: ipmitool -H trial177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-22T06:31:04.087 DEBUG:teuthology.orchestra.console.trial121:pexpect command: ipmitool -H trial121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-22T06:31:04.173 DEBUG:teuthology.orchestra.console.trial127:check power output: Chassis Power is on
2026-02-22T06:31:04.179 DEBUG:teuthology.orchestra.console.trial177:check power output: Chassis Power is on
2026-02-22T06:31:04.197 DEBUG:teuthology.orchestra.console.trial121:check power output: Chassis Power is on
2026-02-22T06:31:08.178 DEBUG:teuthology.orchestra.console.trial127:pexpect command: ipmitool -H trial127.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-22T06:31:08.187 DEBUG:teuthology.orchestra.console.trial177:pexpect command: ipmitool -H trial177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-22T06:31:08.198 DEBUG:teuthology.orchestra.console.trial121:pexpect command: ipmitool -H trial121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-22T06:31:08.289 DEBUG:teuthology.orchestra.console.trial127:check power output: Chassis Power is off
2026-02-22T06:31:08.289 INFO:teuthology.orchestra.console.trial127:Power off completed
2026-02-22T06:31:08.297 DEBUG:teuthology.orchestra.console.trial177:check power output: Chassis Power is off
2026-02-22T06:31:08.297 INFO:teuthology.orchestra.console.trial177:Power off completed
2026-02-22T06:31:08.308 DEBUG:teuthology.orchestra.console.trial121:check power output: Chassis Power is off
2026-02-22T06:31:08.308 INFO:teuthology.orchestra.console.trial121:Power off completed
2026-02-22T06:31:08.582 INFO:teuthology.lock.ops:unlocked: trial127.front.sepia.ceph.com
2026-02-22T06:31:08.764 INFO:teuthology.lock.ops:unlocked: trial177.front.sepia.ceph.com
2026-02-22T06:31:08.772 INFO:teuthology.lock.ops:unlocked: trial121.front.sepia.ceph.com