2026-02-24T05:09:09.590 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-24T05:09:09.590 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:09:09.660 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:09:10.072 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7251f05b9e50>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7251effed6c0>, signals=[15])
2026-02-24T05:09:10.072 INFO:teuthology.lock.ops:Start node 'trial041.front.sepia.ceph.com' reimaging
2026-02-24T05:09:10.072 INFO:teuthology.lock.ops:Updating [trial041.front.sepia.ceph.com]: reset os type and version on server
2026-02-24T05:09:10.072 INFO:teuthology.lock.ops:Updating trial041.front.sepia.ceph.com on lock server
2026-02-24T05:09:10.128 INFO:teuthology.lock.ops:Start node 'trial088.front.sepia.ceph.com' reimaging
2026-02-24T05:09:10.128 INFO:teuthology.lock.ops:Updating [trial088.front.sepia.ceph.com]: reset os type and version on server
2026-02-24T05:09:10.128 INFO:teuthology.lock.ops:Updating trial088.front.sepia.ceph.com on lock server
2026-02-24T05:09:10.184 INFO:teuthology.lock.ops:Start node 'trial167.front.sepia.ceph.com' reimaging
2026-02-24T05:09:10.185 INFO:teuthology.lock.ops:Updating [trial167.front.sepia.ceph.com]: reset os type and version on server
2026-02-24T05:09:10.185 INFO:teuthology.lock.ops:Updating trial167.front.sepia.ceph.com on lock server
2026-02-24T05:09:10.796 DEBUG:teuthology.provision.fog.trial041:Requesting image trial_centos_9.stream (ID 3)
2026-02-24T05:09:10.807 DEBUG:teuthology.provision.fog.trial088:Requesting image trial_centos_9.stream (ID 3)
2026-02-24T05:09:10.813 DEBUG:teuthology.provision.fog.trial167:Requesting image trial_centos_9.stream (ID 3)
2026-02-24T05:09:10.965 INFO:teuthology.provision.fog.trial041:Scheduling deploy of centos 9.stream
2026-02-24T05:09:10.970 INFO:teuthology.provision.fog.trial167:Scheduling deploy of centos 9.stream
2026-02-24T05:09:10.972 INFO:teuthology.provision.fog.trial088:Scheduling deploy of centos 9.stream
2026-02-24T05:09:20.026 INFO:teuthology.orchestra.console.trial088:Power off
2026-02-24T05:09:20.026 DEBUG:teuthology.orchestra.console.trial088:pexpect command: ipmitool -H trial088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T05:09:20.065 DEBUG:teuthology.orchestra.console.trial088:power off output: Chassis Power Control: Down/Off
2026-02-24T05:09:20.219 INFO:teuthology.orchestra.console.trial167:Power off
2026-02-24T05:09:20.220 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T05:09:20.261 DEBUG:teuthology.orchestra.console.trial167:power off output: Chassis Power Control: Down/Off
2026-02-24T05:09:20.567 INFO:teuthology.orchestra.console.trial041:Power off
2026-02-24T05:09:20.567 DEBUG:teuthology.orchestra.console.trial041:pexpect command: ipmitool -H trial041.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T05:09:20.605 DEBUG:teuthology.orchestra.console.trial041:power off output: Chassis Power Control: Down/Off
2026-02-24T05:09:24.069 DEBUG:teuthology.orchestra.console.trial088:pexpect command: ipmitool -H trial088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T05:09:24.178 DEBUG:teuthology.orchestra.console.trial088:check power output: Chassis Power is off
2026-02-24T05:09:24.179 INFO:teuthology.orchestra.console.trial088:Power off completed
2026-02-24T05:09:24.261 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T05:09:24.279 INFO:teuthology.orchestra.console.trial088:Power on
2026-02-24T05:09:24.280 DEBUG:teuthology.orchestra.console.trial088:pexpect command: ipmitool -H trial088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-24T05:09:24.319 DEBUG:teuthology.orchestra.console.trial088:power on output: Chassis Power Control: Up/On
2026-02-24T05:09:24.371 DEBUG:teuthology.orchestra.console.trial167:check power output: Chassis Power is off
2026-02-24T05:09:24.371 INFO:teuthology.orchestra.console.trial167:Power off completed
2026-02-24T05:09:24.472 INFO:teuthology.orchestra.console.trial167:Power on
2026-02-24T05:09:24.472 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-24T05:09:24.513 DEBUG:teuthology.orchestra.console.trial167:power on output: Chassis Power Control: Up/On
2026-02-24T05:09:24.606 DEBUG:teuthology.orchestra.console.trial041:pexpect command: ipmitool -H trial041.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T05:09:24.716 DEBUG:teuthology.orchestra.console.trial041:check power output: Chassis Power is off
2026-02-24T05:09:24.716 INFO:teuthology.orchestra.console.trial041:Power off completed
2026-02-24T05:09:24.816 INFO:teuthology.orchestra.console.trial041:Power on
2026-02-24T05:09:24.816 DEBUG:teuthology.orchestra.console.trial041:pexpect command: ipmitool -H trial041.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-24T05:09:24.859 DEBUG:teuthology.orchestra.console.trial041:power on output: Chassis Power Control: Up/On
2026-02-24T05:09:28.323 DEBUG:teuthology.orchestra.console.trial088:pexpect command: ipmitool -H trial088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T05:09:28.435 DEBUG:teuthology.orchestra.console.trial088:check power output: Chassis Power is on
2026-02-24T05:09:28.435 INFO:teuthology.orchestra.console.trial088:Power on completed
2026-02-24T05:09:28.514 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T05:09:28.536 INFO:teuthology.provision.fog.trial088:Waiting for deploy to finish
2026-02-24T05:09:28.625 DEBUG:teuthology.orchestra.console.trial167:check power output: Chassis Power is on
2026-02-24T05:09:28.625 INFO:teuthology.orchestra.console.trial167:Power on completed
2026-02-24T05:09:28.725 INFO:teuthology.provision.fog.trial167:Waiting for deploy to finish
2026-02-24T05:09:28.860 DEBUG:teuthology.orchestra.console.trial041:pexpect command: ipmitool -H trial041.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T05:09:28.970 DEBUG:teuthology.orchestra.console.trial041:check power output: Chassis Power is on
2026-02-24T05:09:28.971 INFO:teuthology.orchestra.console.trial041:Power on completed
2026-02-24T05:09:29.071 INFO:teuthology.provision.fog.trial041:Waiting for deploy to finish
2026-02-24T05:13:06.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T05:13:09.568 WARNING:teuthology.provision.fog.trial167:[Errno None] Unable to connect to port 22 on 10.20.193.167
2026-02-24T05:13:15.572 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T05:13:18.656 WARNING:teuthology.provision.fog.trial167:[Errno None] Unable to connect to port 22 on 10.20.193.167
2026-02-24T05:13:24.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T05:13:27.744 WARNING:teuthology.provision.fog.trial167:[Errno None] Unable to connect to port 22 on 10.20.193.167
2026-02-24T05:13:33.751 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T05:13:36.832 WARNING:teuthology.provision.fog.trial167:[Errno None] Unable to connect to port 22 on 10.20.193.167
2026-02-24T05:13:42.837 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T05:13:45.921 WARNING:teuthology.provision.fog.trial167:[Errno None] Unable to connect to port 22 on 10.20.193.167
2026-02-24T05:13:51.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T05:14:09.705 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial041.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T05:14:12.800 WARNING:teuthology.provision.fog.trial041:[Errno None] Unable to connect to port 22 on 10.20.193.41
2026-02-24T05:14:18.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial041.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T05:14:21.888 WARNING:teuthology.provision.fog.trial041:[Errno None] Unable to connect to port 22 on 10.20.193.41
2026-02-24T05:14:24.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T05:14:24.420 ERROR:teuthology.orchestra.connection:Error authenticating with trial088.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2026-02-24T05:14:27.660 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-24T05:14:27.661 DEBUG:teuthology.orchestra.run.trial167:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-24T05:14:27.836 INFO:teuthology.provision.fog.trial167:Node is ready
2026-02-24T05:14:27.837 DEBUG:teuthology.orchestra.run.trial167:> hostname
2026-02-24T05:14:27.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial041.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T05:14:27.893 INFO:teuthology.orchestra.run.trial167.stdout:trial167.front.sepia.ceph.com
2026-02-24T05:14:27.894 DEBUG:teuthology.orchestra.run.trial167:> grep trial167.front.sepia.ceph.com /etc/hosts
2026-02-24T05:14:27.951 INFO:teuthology.orchestra.run.trial167.stdout:10.20.193.167 trial167.front.sepia.ceph.com trial167
2026-02-24T05:14:27.951 DEBUG:teuthology.orchestra.run.trial167:> sudo hostname trial167
2026-02-24T05:14:28.022 DEBUG:teuthology.orchestra.run.trial167:> sudo sed -i -e 's/trial167.front.sepia.ceph.com/trial167/g' /etc/hosts
2026-02-24T05:14:28.087 DEBUG:teuthology.orchestra.run.trial167:> sudo sed -i -e 's/10.20.193.167/10.20.193.167/g' /etc/hosts
2026-02-24T05:14:28.151 DEBUG:teuthology.orchestra.run.trial167:> sudo sed -i -e 's/trial167.front.sepia.ceph.com/trial167/g' /etc/hostname
2026-02-24T05:14:28.215 DEBUG:teuthology.orchestra.run.trial167:> sudo hostname trial167
2026-02-24T05:14:28.279 DEBUG:teuthology.orchestra.run.trial167:> cat /etc/os-release
2026-02-24T05:14:28.334 INFO:teuthology.orchestra.run.trial167.stdout:NAME="CentOS Stream"
2026-02-24T05:14:28.334 INFO:teuthology.orchestra.run.trial167.stdout:VERSION="9"
2026-02-24T05:14:28.334 INFO:teuthology.orchestra.run.trial167.stdout:ID="centos"
2026-02-24T05:14:28.334 INFO:teuthology.orchestra.run.trial167.stdout:ID_LIKE="rhel fedora"
2026-02-24T05:14:28.335 INFO:teuthology.orchestra.run.trial167.stdout:VERSION_ID="9"
2026-02-24T05:14:28.335 INFO:teuthology.orchestra.run.trial167.stdout:PLATFORM_ID="platform:el9"
2026-02-24T05:14:28.335 INFO:teuthology.orchestra.run.trial167.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T05:14:28.335 INFO:teuthology.orchestra.run.trial167.stdout:ANSI_COLOR="0;31"
2026-02-24T05:14:28.335 INFO:teuthology.orchestra.run.trial167.stdout:LOGO="fedora-logo-icon"
2026-02-24T05:14:28.335 INFO:teuthology.orchestra.run.trial167.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T05:14:28.335 INFO:teuthology.orchestra.run.trial167.stdout:HOME_URL="https://centos.org/"
2026-02-24T05:14:28.335 INFO:teuthology.orchestra.run.trial167.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T05:14:28.335 INFO:teuthology.orchestra.run.trial167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T05:14:28.335 INFO:teuthology.orchestra.run.trial167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T05:14:28.335 INFO:teuthology.provision.fog.trial167:Deploy complete!
2026-02-24T05:14:28.336 DEBUG:teuthology.parallel:result is None
2026-02-24T05:14:30.976 WARNING:teuthology.provision.fog.trial041:[Errno None] Unable to connect to port 22 on 10.20.193.41
2026-02-24T05:14:36.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial041.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T05:14:40.064 WARNING:teuthology.provision.fog.trial041:[Errno None] Unable to connect to port 22 on 10.20.193.41
2026-02-24T05:14:46.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial041.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T05:14:49.152 WARNING:teuthology.provision.fog.trial041:[Errno None] Unable to connect to port 22 on 10.20.193.41
2026-02-24T05:14:55.153 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial041.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T05:15:28.422 WARNING:teuthology.provision.fog.trial088:timed out
2026-02-24T05:15:31.193 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-24T05:15:31.193 DEBUG:teuthology.orchestra.run.trial041:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-24T05:15:31.361 INFO:teuthology.provision.fog.trial041:Node is ready
2026-02-24T05:15:31.361 DEBUG:teuthology.orchestra.run.trial041:> hostname
2026-02-24T05:15:31.417 INFO:teuthology.orchestra.run.trial041.stdout:trial041.front.sepia.ceph.com
2026-02-24T05:15:31.418 DEBUG:teuthology.orchestra.run.trial041:> grep trial041.front.sepia.ceph.com /etc/hosts
2026-02-24T05:15:31.475 INFO:teuthology.orchestra.run.trial041.stdout:10.20.193.41 trial041.front.sepia.ceph.com trial041
2026-02-24T05:15:31.475 DEBUG:teuthology.orchestra.run.trial041:> sudo hostname trial041
2026-02-24T05:15:31.544 DEBUG:teuthology.orchestra.run.trial041:> sudo sed -i -e 's/trial041.front.sepia.ceph.com/trial041/g' /etc/hosts
2026-02-24T05:15:31.608 DEBUG:teuthology.orchestra.run.trial041:> sudo sed -i -e 's/10.20.193.41/10.20.193.41/g' /etc/hosts
2026-02-24T05:15:31.673 DEBUG:teuthology.orchestra.run.trial041:> sudo sed -i -e 's/trial041.front.sepia.ceph.com/trial041/g' /etc/hostname
2026-02-24T05:15:31.738 DEBUG:teuthology.orchestra.run.trial041:> sudo hostname trial041
2026-02-24T05:15:31.801 DEBUG:teuthology.orchestra.run.trial041:> cat /etc/os-release
2026-02-24T05:15:31.858 INFO:teuthology.orchestra.run.trial041.stdout:NAME="CentOS Stream"
2026-02-24T05:15:31.859 INFO:teuthology.orchestra.run.trial041.stdout:VERSION="9"
2026-02-24T05:15:31.859 INFO:teuthology.orchestra.run.trial041.stdout:ID="centos"
2026-02-24T05:15:31.859 INFO:teuthology.orchestra.run.trial041.stdout:ID_LIKE="rhel fedora"
2026-02-24T05:15:31.859 INFO:teuthology.orchestra.run.trial041.stdout:VERSION_ID="9"
2026-02-24T05:15:31.859 INFO:teuthology.orchestra.run.trial041.stdout:PLATFORM_ID="platform:el9"
2026-02-24T05:15:31.859 INFO:teuthology.orchestra.run.trial041.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T05:15:31.859 INFO:teuthology.orchestra.run.trial041.stdout:ANSI_COLOR="0;31"
2026-02-24T05:15:31.859 INFO:teuthology.orchestra.run.trial041.stdout:LOGO="fedora-logo-icon"
2026-02-24T05:15:31.859 INFO:teuthology.orchestra.run.trial041.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T05:15:31.860 INFO:teuthology.orchestra.run.trial041.stdout:HOME_URL="https://centos.org/"
2026-02-24T05:15:31.860 INFO:teuthology.orchestra.run.trial041.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T05:15:31.860 INFO:teuthology.orchestra.run.trial041.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T05:15:31.860 INFO:teuthology.orchestra.run.trial041.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T05:15:31.860 INFO:teuthology.provision.fog.trial041:Deploy complete!
2026-02-24T05:15:31.861 DEBUG:teuthology.parallel:result is None
2026-02-24T05:15:34.424 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T05:15:36.644 ERROR:teuthology.orchestra.connection:Error authenticating with trial088.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-24T05:15:40.771 ERROR:teuthology.orchestra.connection:Error authenticating with trial088.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-24T05:15:47.893 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-24T05:15:47.893 DEBUG:teuthology.orchestra.run.trial088:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-24T05:15:48.097 INFO:teuthology.provision.fog.trial088:Node is ready
2026-02-24T05:15:48.098 DEBUG:teuthology.orchestra.run.trial088:> hostname
2026-02-24T05:15:48.154 INFO:teuthology.orchestra.run.trial088.stdout:trial088.front.sepia.ceph.com
2026-02-24T05:15:48.155 DEBUG:teuthology.orchestra.run.trial088:> grep trial088.front.sepia.ceph.com /etc/hosts
2026-02-24T05:15:48.212 INFO:teuthology.orchestra.run.trial088.stdout:10.20.193.88 trial088.front.sepia.ceph.com trial088
2026-02-24T05:15:48.212 DEBUG:teuthology.orchestra.run.trial088:> sudo hostname trial088
2026-02-24T05:15:48.282 DEBUG:teuthology.orchestra.run.trial088:> sudo sed -i -e 's/trial088.front.sepia.ceph.com/trial088/g' /etc/hosts
2026-02-24T05:15:48.349 DEBUG:teuthology.orchestra.run.trial088:> sudo sed -i -e 's/10.20.193.88/10.20.193.88/g' /etc/hosts
2026-02-24T05:15:48.413 DEBUG:teuthology.orchestra.run.trial088:> sudo sed -i -e 's/trial088.front.sepia.ceph.com/trial088/g' /etc/hostname
2026-02-24T05:15:48.479 DEBUG:teuthology.orchestra.run.trial088:> sudo hostname trial088
2026-02-24T05:15:48.545 DEBUG:teuthology.orchestra.run.trial088:> cat /etc/os-release
2026-02-24T05:15:48.603 INFO:teuthology.orchestra.run.trial088.stdout:NAME="CentOS Stream"
2026-02-24T05:15:48.604 INFO:teuthology.orchestra.run.trial088.stdout:VERSION="9"
2026-02-24T05:15:48.604 INFO:teuthology.orchestra.run.trial088.stdout:ID="centos"
2026-02-24T05:15:48.604 INFO:teuthology.orchestra.run.trial088.stdout:ID_LIKE="rhel fedora"
2026-02-24T05:15:48.604 INFO:teuthology.orchestra.run.trial088.stdout:VERSION_ID="9"
2026-02-24T05:15:48.604 INFO:teuthology.orchestra.run.trial088.stdout:PLATFORM_ID="platform:el9"
2026-02-24T05:15:48.604 INFO:teuthology.orchestra.run.trial088.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T05:15:48.604 INFO:teuthology.orchestra.run.trial088.stdout:ANSI_COLOR="0;31"
2026-02-24T05:15:48.604 INFO:teuthology.orchestra.run.trial088.stdout:LOGO="fedora-logo-icon"
2026-02-24T05:15:48.604 INFO:teuthology.orchestra.run.trial088.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T05:15:48.604 INFO:teuthology.orchestra.run.trial088.stdout:HOME_URL="https://centos.org/"
2026-02-24T05:15:48.604 INFO:teuthology.orchestra.run.trial088.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T05:15:48.604 INFO:teuthology.orchestra.run.trial088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T05:15:48.604 INFO:teuthology.orchestra.run.trial088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T05:15:48.605 INFO:teuthology.provision.fog.trial088:Deploy complete!
2026-02-24T05:15:48.606 DEBUG:teuthology.parallel:result is None
2026-02-24T05:15:49.333 INFO:teuthology.lock.ops:Checking trial041.front.sepia.ceph.com
2026-02-24T05:15:49.334 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-24T05:15:50.107 INFO:teuthology.lock.ops:Checking trial088.front.sepia.ceph.com
2026-02-24T05:15:50.107 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-24T05:15:50.215 INFO:teuthology.lock.ops:Checking trial167.front.sepia.ceph.com
2026-02-24T05:15:50.215 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-24T05:15:50.272 INFO:teuthology.lock.ops:Updating [trial041.front.sepia.ceph.com]: set os type and version on server
2026-02-24T05:15:50.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial041.front.sepia.ceph.com'
2026-02-24T05:15:50.273 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial041.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T05:15:50.335 DEBUG:teuthology.orchestra.run.trial041:> true
2026-02-24T05:15:50.409 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial041.front.sepia.ceph.com'
2026-02-24T05:15:50.409 DEBUG:teuthology.orchestra.run.trial041:> uname -m
2026-02-24T05:15:50.464 INFO:teuthology.orchestra.run.trial041.stdout:x86_64
2026-02-24T05:15:50.465 DEBUG:teuthology.orchestra.run.trial041:> cat /etc/os-release
2026-02-24T05:15:50.521 INFO:teuthology.orchestra.run.trial041.stdout:NAME="CentOS Stream"
2026-02-24T05:15:50.522 INFO:teuthology.orchestra.run.trial041.stdout:VERSION="9"
2026-02-24T05:15:50.522 INFO:teuthology.orchestra.run.trial041.stdout:ID="centos"
2026-02-24T05:15:50.522 INFO:teuthology.orchestra.run.trial041.stdout:ID_LIKE="rhel fedora"
2026-02-24T05:15:50.522 INFO:teuthology.orchestra.run.trial041.stdout:VERSION_ID="9"
2026-02-24T05:15:50.522 INFO:teuthology.orchestra.run.trial041.stdout:PLATFORM_ID="platform:el9"
2026-02-24T05:15:50.522 INFO:teuthology.orchestra.run.trial041.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T05:15:50.522 INFO:teuthology.orchestra.run.trial041.stdout:ANSI_COLOR="0;31"
2026-02-24T05:15:50.522 INFO:teuthology.orchestra.run.trial041.stdout:LOGO="fedora-logo-icon"
2026-02-24T05:15:50.522 INFO:teuthology.orchestra.run.trial041.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T05:15:50.522 INFO:teuthology.orchestra.run.trial041.stdout:HOME_URL="https://centos.org/"
2026-02-24T05:15:50.523 INFO:teuthology.orchestra.run.trial041.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T05:15:50.523 INFO:teuthology.orchestra.run.trial041.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T05:15:50.523 INFO:teuthology.orchestra.run.trial041.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T05:15:50.524 INFO:teuthology.lock.ops:Updating trial041.front.sepia.ceph.com on lock server
2026-02-24T05:15:50.631 INFO:teuthology.lock.ops:Updating [trial088.front.sepia.ceph.com]: set os type and version on server
2026-02-24T05:15:50.631 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial088.front.sepia.ceph.com'
2026-02-24T05:15:50.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T05:15:50.696 DEBUG:teuthology.orchestra.run.trial088:> true
2026-02-24T05:15:50.774 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial088.front.sepia.ceph.com'
2026-02-24T05:15:50.774 DEBUG:teuthology.orchestra.run.trial088:> uname -m
2026-02-24T05:15:50.830 INFO:teuthology.orchestra.run.trial088.stdout:x86_64
2026-02-24T05:15:50.830 DEBUG:teuthology.orchestra.run.trial088:> cat /etc/os-release
2026-02-24T05:15:50.886 INFO:teuthology.orchestra.run.trial088.stdout:NAME="CentOS Stream"
2026-02-24T05:15:50.886 INFO:teuthology.orchestra.run.trial088.stdout:VERSION="9"
2026-02-24T05:15:50.886 INFO:teuthology.orchestra.run.trial088.stdout:ID="centos"
2026-02-24T05:15:50.886 INFO:teuthology.orchestra.run.trial088.stdout:ID_LIKE="rhel fedora"
2026-02-24T05:15:50.886 INFO:teuthology.orchestra.run.trial088.stdout:VERSION_ID="9"
2026-02-24T05:15:50.886 INFO:teuthology.orchestra.run.trial088.stdout:PLATFORM_ID="platform:el9"
2026-02-24T05:15:50.886 INFO:teuthology.orchestra.run.trial088.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T05:15:50.887 INFO:teuthology.orchestra.run.trial088.stdout:ANSI_COLOR="0;31"
2026-02-24T05:15:50.887 INFO:teuthology.orchestra.run.trial088.stdout:LOGO="fedora-logo-icon"
2026-02-24T05:15:50.887 INFO:teuthology.orchestra.run.trial088.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T05:15:50.887 INFO:teuthology.orchestra.run.trial088.stdout:HOME_URL="https://centos.org/"
2026-02-24T05:15:50.887 INFO:teuthology.orchestra.run.trial088.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T05:15:50.887 INFO:teuthology.orchestra.run.trial088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T05:15:50.887 INFO:teuthology.orchestra.run.trial088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T05:15:50.888 INFO:teuthology.lock.ops:Updating trial088.front.sepia.ceph.com on lock server
2026-02-24T05:15:51.125 INFO:teuthology.lock.ops:Updating [trial167.front.sepia.ceph.com]: set os type and version on server
2026-02-24T05:15:51.126 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial167.front.sepia.ceph.com'
2026-02-24T05:15:51.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T05:15:51.192 DEBUG:teuthology.orchestra.run.trial167:> true
2026-02-24T05:15:51.339 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial167.front.sepia.ceph.com'
2026-02-24T05:15:51.340 DEBUG:teuthology.orchestra.run.trial167:> uname -m
2026-02-24T05:15:51.396 INFO:teuthology.orchestra.run.trial167.stdout:x86_64
2026-02-24T05:15:51.396 DEBUG:teuthology.orchestra.run.trial167:> cat /etc/os-release
2026-02-24T05:15:51.453 INFO:teuthology.orchestra.run.trial167.stdout:NAME="CentOS Stream"
2026-02-24T05:15:51.453 INFO:teuthology.orchestra.run.trial167.stdout:VERSION="9"
2026-02-24T05:15:51.454 INFO:teuthology.orchestra.run.trial167.stdout:ID="centos"
2026-02-24T05:15:51.454 INFO:teuthology.orchestra.run.trial167.stdout:ID_LIKE="rhel fedora"
2026-02-24T05:15:51.454 INFO:teuthology.orchestra.run.trial167.stdout:VERSION_ID="9"
2026-02-24T05:15:51.454 INFO:teuthology.orchestra.run.trial167.stdout:PLATFORM_ID="platform:el9"
2026-02-24T05:15:51.454 INFO:teuthology.orchestra.run.trial167.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T05:15:51.454 INFO:teuthology.orchestra.run.trial167.stdout:ANSI_COLOR="0;31"
2026-02-24T05:15:51.454 INFO:teuthology.orchestra.run.trial167.stdout:LOGO="fedora-logo-icon"
2026-02-24T05:15:51.454 INFO:teuthology.orchestra.run.trial167.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T05:15:51.454 INFO:teuthology.orchestra.run.trial167.stdout:HOME_URL="https://centos.org/"
2026-02-24T05:15:51.454 INFO:teuthology.orchestra.run.trial167.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T05:15:51.454 INFO:teuthology.orchestra.run.trial167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T05:15:51.455 INFO:teuthology.orchestra.run.trial167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T05:15:51.455 INFO:teuthology.lock.ops:Updating trial167.front.sepia.ceph.com on lock server
2026-02-24T05:15:56.934 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:15:57.027 INFO:teuthology.dispatcher.supervisor:Running job 66991
2026-02-24T05:15:57.027 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/bin/teuthology -v --owner scheduled_vshankar@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66991 --name vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial --description fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate 3-kernel} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn pg_health} tasks/{1-tests/fscrypt-ffsb}} -- /home/teuthworker/mnt/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66991/orig.config.yaml
2026-02-24T05:15:57.027 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66991
2026-02-24T05:15:57.027 INFO:teuthology.dispatcher.supervisor:Job PID: 2086287
2026-02-24T05:15:57.027 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-24T05:17:57.028 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:19:57.148 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:21:57.244 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:23:57.341 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:25:57.462 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:27:57.789 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:29:57.910 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:31:58.043 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:33:58.162 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:35:58.277 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:37:58.397 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:39:58.516 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:41:58.632 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:43:58.721 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:45:58.841 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:47:58.957 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:50:01.295 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:52:01.402 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:54:01.522 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:56:01.643 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T05:58:01.767 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T06:00:01.860 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T06:02:01.975 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T06:04:02.097 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T06:06:02.215 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T06:08:02.336 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T06:08:02.459 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-24T06:08:02.662 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-24T06:08:03.374 INFO:teuthology.orchestra.console.trial088:Power off
2026-02-24T06:08:03.374 DEBUG:teuthology.orchestra.console.trial088:pexpect command: ipmitool -H trial088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T06:08:03.395 INFO:teuthology.orchestra.console.trial167:Power off
2026-02-24T06:08:03.395 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T06:08:03.406 INFO:teuthology.orchestra.console.trial041:Power off
2026-02-24T06:08:03.406 DEBUG:teuthology.orchestra.console.trial041:pexpect command: ipmitool -H trial041.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T06:08:03.412 DEBUG:teuthology.orchestra.console.trial088:power off output: Chassis Power Control: Down/Off
2026-02-24T06:08:03.434 DEBUG:teuthology.orchestra.console.trial167:power off output: Chassis Power Control: Down/Off
2026-02-24T06:08:03.446 DEBUG:teuthology.orchestra.console.trial041:power off output: Chassis Power Control: Down/Off
2026-02-24T06:08:07.413 DEBUG:teuthology.orchestra.console.trial088:pexpect command: ipmitool -H trial088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T06:08:07.434 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T06:08:07.447 DEBUG:teuthology.orchestra.console.trial041:pexpect command: ipmitool -H trial041.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T06:08:07.524 DEBUG:teuthology.orchestra.console.trial088:check power output: Chassis Power is on
2026-02-24T06:08:07.544 DEBUG:teuthology.orchestra.console.trial167:check power output: Chassis Power is on
2026-02-24T06:08:07.555 DEBUG:teuthology.orchestra.console.trial041:check power output: Chassis Power is on
2026-02-24T06:08:11.525 DEBUG:teuthology.orchestra.console.trial088:pexpect command: ipmitool -H trial088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T06:08:11.545 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T06:08:11.557 DEBUG:teuthology.orchestra.console.trial041:pexpect command: ipmitool -H trial041.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T06:08:11.635 DEBUG:teuthology.orchestra.console.trial088:check power output: Chassis Power is off
2026-02-24T06:08:11.635 INFO:teuthology.orchestra.console.trial088:Power off completed
2026-02-24T06:08:11.655 DEBUG:teuthology.orchestra.console.trial167:check power output: Chassis Power is off
2026-02-24T06:08:11.655 INFO:teuthology.orchestra.console.trial167:Power off completed
2026-02-24T06:08:11.666 DEBUG:teuthology.orchestra.console.trial041:check power output: Chassis Power is off
2026-02-24T06:08:11.666 INFO:teuthology.orchestra.console.trial041:Power off completed
2026-02-24T06:08:11.934 INFO:teuthology.lock.ops:unlocked: trial088.front.sepia.ceph.com
2026-02-24T06:08:11.945 INFO:teuthology.lock.ops:unlocked: trial167.front.sepia.ceph.com
2026-02-24T06:08:11.949 INFO:teuthology.lock.ops:unlocked: trial041.front.sepia.ceph.com