2026-02-24T09:25:59.409 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-24T09:25:59.409 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:25:59.527 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:26:02.880 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7ef4153a3b90>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7ef414b52d40>, signals=[15])
2026-02-24T09:26:02.880 INFO:teuthology.lock.ops:Start node 'trial020.front.sepia.ceph.com' reimaging
2026-02-24T09:26:02.880 INFO:teuthology.lock.ops:Updating [trial020.front.sepia.ceph.com]: reset os type and version on server
2026-02-24T09:26:02.880 INFO:teuthology.lock.ops:Updating trial020.front.sepia.ceph.com on lock server
2026-02-24T09:26:02.939 INFO:teuthology.lock.ops:Start node 'trial036.front.sepia.ceph.com' reimaging
2026-02-24T09:26:02.939 INFO:teuthology.lock.ops:Updating [trial036.front.sepia.ceph.com]: reset os type and version on server
2026-02-24T09:26:02.939 INFO:teuthology.lock.ops:Updating trial036.front.sepia.ceph.com on lock server
2026-02-24T09:26:03.056 INFO:teuthology.lock.ops:Start node 'trial158.front.sepia.ceph.com' reimaging
2026-02-24T09:26:03.057 INFO:teuthology.lock.ops:Updating [trial158.front.sepia.ceph.com]: reset os type and version on server
2026-02-24T09:26:03.057 INFO:teuthology.lock.ops:Updating trial158.front.sepia.ceph.com on lock server
2026-02-24T09:26:03.185 DEBUG:teuthology.provision.fog.trial020:Requesting image trial_centos_9.stream (ID 3)
2026-02-24T09:26:03.237 INFO:teuthology.provision.fog.trial020:Scheduling deploy of centos 9.stream
2026-02-24T09:26:03.281 DEBUG:teuthology.provision.fog.trial036:Requesting image trial_centos_9.stream (ID 3)
2026-02-24T09:26:03.287 DEBUG:teuthology.provision.fog.trial158:Requesting image trial_centos_9.stream (ID 3)
2026-02-24T09:26:03.335 INFO:teuthology.provision.fog.trial036:Scheduling deploy of centos 9.stream
2026-02-24T09:26:03.338 INFO:teuthology.provision.fog.trial158:Scheduling deploy of centos 9.stream
2026-02-24T09:26:04.821 INFO:teuthology.orchestra.console.trial020:Power off
2026-02-24T09:26:04.821 DEBUG:teuthology.orchestra.console.trial020:pexpect command: ipmitool -H trial020.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T09:26:04.861 DEBUG:teuthology.orchestra.console.trial020:power off output: Chassis Power Control: Down/Off
2026-02-24T09:26:05.119 INFO:teuthology.orchestra.console.trial158:Power off
2026-02-24T09:26:05.119 DEBUG:teuthology.orchestra.console.trial158:pexpect command: ipmitool -H trial158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T09:26:05.177 DEBUG:teuthology.orchestra.console.trial158:power off output: Chassis Power Control: Down/Off
2026-02-24T09:26:05.189 INFO:teuthology.orchestra.console.trial036:Power off
2026-02-24T09:26:05.189 DEBUG:teuthology.orchestra.console.trial036:pexpect command: ipmitool -H trial036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T09:26:05.234 DEBUG:teuthology.orchestra.console.trial036:power off output: Chassis Power Control: Down/Off
2026-02-24T09:26:08.865 DEBUG:teuthology.orchestra.console.trial020:pexpect command: ipmitool -H trial020.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T09:26:08.974 DEBUG:teuthology.orchestra.console.trial020:check power output: Chassis Power is off
2026-02-24T09:26:08.975 INFO:teuthology.orchestra.console.trial020:Power off completed
2026-02-24T09:26:09.075 INFO:teuthology.orchestra.console.trial020:Power on
2026-02-24T09:26:09.075 DEBUG:teuthology.orchestra.console.trial020:pexpect command: ipmitool -H trial020.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-24T09:26:09.117 DEBUG:teuthology.orchestra.console.trial020:power on output: Chassis Power Control: Up/On
2026-02-24T09:26:09.178 DEBUG:teuthology.orchestra.console.trial158:pexpect command: ipmitool -H trial158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T09:26:09.235 DEBUG:teuthology.orchestra.console.trial036:pexpect command: ipmitool -H trial036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T09:26:09.288 DEBUG:teuthology.orchestra.console.trial158:check power output: Chassis Power is off
2026-02-24T09:26:09.289 INFO:teuthology.orchestra.console.trial158:Power off completed
2026-02-24T09:26:09.343 DEBUG:teuthology.orchestra.console.trial036:check power output: Chassis Power is off
2026-02-24T09:26:09.343 INFO:teuthology.orchestra.console.trial036:Power off completed
2026-02-24T09:26:09.389 INFO:teuthology.orchestra.console.trial158:Power on
2026-02-24T09:26:09.390 DEBUG:teuthology.orchestra.console.trial158:pexpect command: ipmitool -H trial158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-24T09:26:09.438 DEBUG:teuthology.orchestra.console.trial158:power on output: Chassis Power Control: Up/On
2026-02-24T09:26:09.444 INFO:teuthology.orchestra.console.trial036:Power on
2026-02-24T09:26:09.444 DEBUG:teuthology.orchestra.console.trial036:pexpect command: ipmitool -H trial036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-24T09:26:09.484 DEBUG:teuthology.orchestra.console.trial036:power on output: Chassis Power Control: Up/On
2026-02-24T09:26:13.120 DEBUG:teuthology.orchestra.console.trial020:pexpect command: ipmitool -H trial020.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T09:26:13.230 DEBUG:teuthology.orchestra.console.trial020:check power output: Chassis Power is on
2026-02-24T09:26:13.230 INFO:teuthology.orchestra.console.trial020:Power on completed
2026-02-24T09:26:13.331 INFO:teuthology.provision.fog.trial020:Waiting for deploy to finish
2026-02-24T09:26:13.439 DEBUG:teuthology.orchestra.console.trial158:pexpect command: ipmitool -H trial158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T09:26:13.485 DEBUG:teuthology.orchestra.console.trial036:pexpect command: ipmitool -H trial036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T09:26:13.550 DEBUG:teuthology.orchestra.console.trial158:check power output: Chassis Power is on
2026-02-24T09:26:13.550 INFO:teuthology.orchestra.console.trial158:Power on completed
2026-02-24T09:26:13.593 DEBUG:teuthology.orchestra.console.trial036:check power output: Chassis Power is on
2026-02-24T09:26:13.593 INFO:teuthology.orchestra.console.trial036:Power on completed
2026-02-24T09:26:13.651 INFO:teuthology.provision.fog.trial158:Waiting for deploy to finish
2026-02-24T09:26:13.694 INFO:teuthology.provision.fog.trial036:Waiting for deploy to finish
2026-02-24T09:28:01.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial020.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T09:28:01.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T09:28:01.981 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T09:28:02.128 ERROR:teuthology.orchestra.connection:Error authenticating with trial158.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2026-02-24T09:28:04.288 WARNING:teuthology.provision.fog.trial020:[Errno None] Unable to connect to port 22 on 10.20.193.20
2026-02-24T09:28:05.056 WARNING:teuthology.provision.fog.trial036:[Errno None] Unable to connect to port 22 on 10.20.193.36
2026-02-24T09:28:10.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial020.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T09:28:11.056 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T09:28:13.376 WARNING:teuthology.provision.fog.trial020:[Errno None] Unable to connect to port 22 on 10.20.193.20
2026-02-24T09:28:14.144 WARNING:teuthology.provision.fog.trial036:[Errno None] Unable to connect to port 22 on 10.20.193.36
2026-02-24T09:28:19.377 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial020.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T09:28:20.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T09:28:22.464 WARNING:teuthology.provision.fog.trial020:[Errno None] Unable to connect to port 22 on 10.20.193.20
2026-02-24T09:28:23.232 WARNING:teuthology.provision.fog.trial036:[Errno None] Unable to connect to port 22 on 10.20.193.36
2026-02-24T09:28:28.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial020.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T09:28:29.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T09:28:31.552 WARNING:teuthology.provision.fog.trial020:[Errno None] Unable to connect to port 22 on 10.20.193.20
2026-02-24T09:28:32.320 WARNING:teuthology.provision.fog.trial036:[Errno None] Unable to connect to port 22 on 10.20.193.36
2026-02-24T09:28:37.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial020.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T09:28:38.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T09:28:40.640 WARNING:teuthology.provision.fog.trial020:[Errno None] Unable to connect to port 22 on 10.20.193.20
2026-02-24T09:28:41.407 WARNING:teuthology.provision.fog.trial036:[Errno None] Unable to connect to port 22 on 10.20.193.36
2026-02-24T09:28:46.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial020.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T09:28:47.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T09:29:06.131 WARNING:teuthology.provision.fog.trial158:timed out
2026-02-24T09:29:12.132 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T09:29:14.312 ERROR:teuthology.orchestra.connection:Error authenticating with trial158.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-24T09:29:18.417 ERROR:teuthology.orchestra.connection:Error authenticating with trial158.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-24T09:29:22.678 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-24T09:29:22.679 DEBUG:teuthology.orchestra.run.trial020:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-24T09:29:22.899 INFO:teuthology.provision.fog.trial020:Node is ready
2026-02-24T09:29:22.899 DEBUG:teuthology.orchestra.run.trial020:> hostname
2026-02-24T09:29:22.954 INFO:teuthology.orchestra.run.trial020.stdout:trial020.front.sepia.ceph.com
2026-02-24T09:29:22.955 DEBUG:teuthology.orchestra.run.trial020:> grep trial020.front.sepia.ceph.com /etc/hosts
2026-02-24T09:29:23.013 INFO:teuthology.orchestra.run.trial020.stdout:10.20.193.20 trial020.front.sepia.ceph.com trial020
2026-02-24T09:29:23.014 DEBUG:teuthology.orchestra.run.trial020:> sudo hostname trial020
2026-02-24T09:29:23.084 DEBUG:teuthology.orchestra.run.trial020:> sudo sed -i -e 's/trial020.front.sepia.ceph.com/trial020/g' /etc/hosts
2026-02-24T09:29:23.198 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-24T09:29:23.198 DEBUG:teuthology.orchestra.run.trial036:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-24T09:29:23.199 DEBUG:teuthology.orchestra.run.trial020:> sudo sed -i -e 's/10.20.193.20/10.20.193.20/g' /etc/hosts
2026-02-24T09:29:23.264 DEBUG:teuthology.orchestra.run.trial020:> sudo sed -i -e 's/trial020.front.sepia.ceph.com/trial020/g' /etc/hostname
2026-02-24T09:29:23.329 DEBUG:teuthology.orchestra.run.trial020:> sudo hostname trial020
2026-02-24T09:29:23.394 DEBUG:teuthology.orchestra.run.trial020:> cat /etc/os-release
2026-02-24T09:29:23.416 INFO:teuthology.provision.fog.trial036:Node is ready
2026-02-24T09:29:23.416 DEBUG:teuthology.orchestra.run.trial036:> hostname
2026-02-24T09:29:23.450 INFO:teuthology.orchestra.run.trial020.stdout:NAME="CentOS Stream"
2026-02-24T09:29:23.450 INFO:teuthology.orchestra.run.trial020.stdout:VERSION="9"
2026-02-24T09:29:23.450 INFO:teuthology.orchestra.run.trial020.stdout:ID="centos"
2026-02-24T09:29:23.450 INFO:teuthology.orchestra.run.trial020.stdout:ID_LIKE="rhel fedora"
2026-02-24T09:29:23.450 INFO:teuthology.orchestra.run.trial020.stdout:VERSION_ID="9"
2026-02-24T09:29:23.450 INFO:teuthology.orchestra.run.trial020.stdout:PLATFORM_ID="platform:el9"
2026-02-24T09:29:23.451 INFO:teuthology.orchestra.run.trial020.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T09:29:23.451 INFO:teuthology.orchestra.run.trial020.stdout:ANSI_COLOR="0;31"
2026-02-24T09:29:23.451 INFO:teuthology.orchestra.run.trial020.stdout:LOGO="fedora-logo-icon"
2026-02-24T09:29:23.451 INFO:teuthology.orchestra.run.trial020.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T09:29:23.451 INFO:teuthology.orchestra.run.trial020.stdout:HOME_URL="https://centos.org/"
2026-02-24T09:29:23.451 INFO:teuthology.orchestra.run.trial020.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T09:29:23.451 INFO:teuthology.orchestra.run.trial020.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T09:29:23.451 INFO:teuthology.orchestra.run.trial020.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T09:29:23.452 INFO:teuthology.provision.fog.trial020:Deploy complete!
2026-02-24T09:29:23.453 DEBUG:teuthology.parallel:result is None
2026-02-24T09:29:23.476 INFO:teuthology.orchestra.run.trial036.stdout:trial036.front.sepia.ceph.com
2026-02-24T09:29:23.476 DEBUG:teuthology.orchestra.run.trial036:> grep trial036.front.sepia.ceph.com /etc/hosts
2026-02-24T09:29:23.537 INFO:teuthology.orchestra.run.trial036.stdout:10.20.193.36 trial036.front.sepia.ceph.com trial036
2026-02-24T09:29:23.538 DEBUG:teuthology.orchestra.run.trial036:> sudo hostname trial036
2026-02-24T09:29:23.611 DEBUG:teuthology.orchestra.run.trial036:> sudo sed -i -e 's/trial036.front.sepia.ceph.com/trial036/g' /etc/hosts
2026-02-24T09:29:23.680 DEBUG:teuthology.orchestra.run.trial036:> sudo sed -i -e 's/10.20.193.36/10.20.193.36/g' /etc/hosts
2026-02-24T09:29:23.749 DEBUG:teuthology.orchestra.run.trial036:> sudo sed -i -e 's/trial036.front.sepia.ceph.com/trial036/g' /etc/hostname
2026-02-24T09:29:23.815 DEBUG:teuthology.orchestra.run.trial036:> sudo hostname trial036
2026-02-24T09:29:23.884 DEBUG:teuthology.orchestra.run.trial036:> cat /etc/os-release
2026-02-24T09:29:23.942 INFO:teuthology.orchestra.run.trial036.stdout:NAME="CentOS Stream"
2026-02-24T09:29:23.942 INFO:teuthology.orchestra.run.trial036.stdout:VERSION="9"
2026-02-24T09:29:23.943 INFO:teuthology.orchestra.run.trial036.stdout:ID="centos"
2026-02-24T09:29:23.943 INFO:teuthology.orchestra.run.trial036.stdout:ID_LIKE="rhel fedora"
2026-02-24T09:29:23.943 INFO:teuthology.orchestra.run.trial036.stdout:VERSION_ID="9"
2026-02-24T09:29:23.943 INFO:teuthology.orchestra.run.trial036.stdout:PLATFORM_ID="platform:el9"
2026-02-24T09:29:23.943 INFO:teuthology.orchestra.run.trial036.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T09:29:23.943 INFO:teuthology.orchestra.run.trial036.stdout:ANSI_COLOR="0;31"
2026-02-24T09:29:23.943 INFO:teuthology.orchestra.run.trial036.stdout:LOGO="fedora-logo-icon"
2026-02-24T09:29:23.943 INFO:teuthology.orchestra.run.trial036.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T09:29:23.943 INFO:teuthology.orchestra.run.trial036.stdout:HOME_URL="https://centos.org/"
2026-02-24T09:29:23.943 INFO:teuthology.orchestra.run.trial036.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T09:29:23.943 INFO:teuthology.orchestra.run.trial036.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T09:29:23.944 INFO:teuthology.orchestra.run.trial036.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T09:29:23.944 INFO:teuthology.provision.fog.trial036:Deploy complete!
2026-02-24T09:29:23.945 DEBUG:teuthology.parallel:result is None
2026-02-24T09:29:25.529 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-24T09:29:25.529 DEBUG:teuthology.orchestra.run.trial158:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-24T09:29:25.882 INFO:teuthology.provision.fog.trial158:Node is ready
2026-02-24T09:29:25.882 DEBUG:teuthology.orchestra.run.trial158:> hostname
2026-02-24T09:29:25.939 INFO:teuthology.orchestra.run.trial158.stdout:trial158.front.sepia.ceph.com
2026-02-24T09:29:25.939 DEBUG:teuthology.orchestra.run.trial158:> grep trial158.front.sepia.ceph.com /etc/hosts
2026-02-24T09:29:25.996 INFO:teuthology.orchestra.run.trial158.stdout:10.20.193.158 trial158.front.sepia.ceph.com trial158
2026-02-24T09:29:25.996 DEBUG:teuthology.orchestra.run.trial158:> sudo hostname trial158
2026-02-24T09:29:26.066 DEBUG:teuthology.orchestra.run.trial158:> sudo sed -i -e 's/trial158.front.sepia.ceph.com/trial158/g' /etc/hosts
2026-02-24T09:29:26.132 DEBUG:teuthology.orchestra.run.trial158:> sudo sed -i -e 's/10.20.193.158/10.20.193.158/g' /etc/hosts
2026-02-24T09:29:26.199 DEBUG:teuthology.orchestra.run.trial158:> sudo sed -i -e 's/trial158.front.sepia.ceph.com/trial158/g' /etc/hostname
2026-02-24T09:29:26.265 DEBUG:teuthology.orchestra.run.trial158:> sudo hostname trial158
2026-02-24T09:29:26.329 DEBUG:teuthology.orchestra.run.trial158:> cat /etc/os-release
2026-02-24T09:29:26.384 INFO:teuthology.orchestra.run.trial158.stdout:NAME="CentOS Stream"
2026-02-24T09:29:26.385 INFO:teuthology.orchestra.run.trial158.stdout:VERSION="9"
2026-02-24T09:29:26.385 INFO:teuthology.orchestra.run.trial158.stdout:ID="centos"
2026-02-24T09:29:26.385 INFO:teuthology.orchestra.run.trial158.stdout:ID_LIKE="rhel fedora"
2026-02-24T09:29:26.385 INFO:teuthology.orchestra.run.trial158.stdout:VERSION_ID="9"
2026-02-24T09:29:26.385 INFO:teuthology.orchestra.run.trial158.stdout:PLATFORM_ID="platform:el9"
2026-02-24T09:29:26.385 INFO:teuthology.orchestra.run.trial158.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T09:29:26.385 INFO:teuthology.orchestra.run.trial158.stdout:ANSI_COLOR="0;31"
2026-02-24T09:29:26.385 INFO:teuthology.orchestra.run.trial158.stdout:LOGO="fedora-logo-icon"
2026-02-24T09:29:26.385 INFO:teuthology.orchestra.run.trial158.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T09:29:26.386 INFO:teuthology.orchestra.run.trial158.stdout:HOME_URL="https://centos.org/"
2026-02-24T09:29:26.386 INFO:teuthology.orchestra.run.trial158.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T09:29:26.386 INFO:teuthology.orchestra.run.trial158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T09:29:26.386 INFO:teuthology.orchestra.run.trial158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T09:29:26.386 INFO:teuthology.provision.fog.trial158:Deploy complete!
2026-02-24T09:29:26.387 DEBUG:teuthology.parallel:result is None
2026-02-24T09:29:26.897 INFO:teuthology.lock.ops:Checking trial020.front.sepia.ceph.com
2026-02-24T09:29:26.897 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-24T09:29:27.004 INFO:teuthology.lock.ops:Checking trial036.front.sepia.ceph.com
2026-02-24T09:29:27.004 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-24T09:29:27.072 INFO:teuthology.lock.ops:Checking trial158.front.sepia.ceph.com
2026-02-24T09:29:27.073 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-24T09:29:27.130 INFO:teuthology.lock.ops:Updating [trial020.front.sepia.ceph.com]: set os type and version on server
2026-02-24T09:29:27.130 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial020.front.sepia.ceph.com'
2026-02-24T09:29:27.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial020.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T09:29:27.189 DEBUG:teuthology.orchestra.run.trial020:> true
2026-02-24T09:29:27.266 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial020.front.sepia.ceph.com'
2026-02-24T09:29:27.266 DEBUG:teuthology.orchestra.run.trial020:> uname -m
2026-02-24T09:29:27.322 INFO:teuthology.orchestra.run.trial020.stdout:x86_64
2026-02-24T09:29:27.323 DEBUG:teuthology.orchestra.run.trial020:> cat /etc/os-release
2026-02-24T09:29:27.378 INFO:teuthology.orchestra.run.trial020.stdout:NAME="CentOS Stream"
2026-02-24T09:29:27.379 INFO:teuthology.orchestra.run.trial020.stdout:VERSION="9"
2026-02-24T09:29:27.379 INFO:teuthology.orchestra.run.trial020.stdout:ID="centos"
2026-02-24T09:29:27.379 INFO:teuthology.orchestra.run.trial020.stdout:ID_LIKE="rhel fedora"
2026-02-24T09:29:27.379 INFO:teuthology.orchestra.run.trial020.stdout:VERSION_ID="9"
2026-02-24T09:29:27.379 INFO:teuthology.orchestra.run.trial020.stdout:PLATFORM_ID="platform:el9"
2026-02-24T09:29:27.379 INFO:teuthology.orchestra.run.trial020.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T09:29:27.379 INFO:teuthology.orchestra.run.trial020.stdout:ANSI_COLOR="0;31"
2026-02-24T09:29:27.379 INFO:teuthology.orchestra.run.trial020.stdout:LOGO="fedora-logo-icon"
2026-02-24T09:29:27.379 INFO:teuthology.orchestra.run.trial020.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T09:29:27.379 INFO:teuthology.orchestra.run.trial020.stdout:HOME_URL="https://centos.org/"
2026-02-24T09:29:27.379 INFO:teuthology.orchestra.run.trial020.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T09:29:27.380 INFO:teuthology.orchestra.run.trial020.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T09:29:27.380 INFO:teuthology.orchestra.run.trial020.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T09:29:27.380 INFO:teuthology.lock.ops:Updating trial020.front.sepia.ceph.com on lock server
2026-02-24T09:29:27.484 INFO:teuthology.lock.ops:Updating [trial036.front.sepia.ceph.com]: set os type and version on server
2026-02-24T09:29:27.484 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial036.front.sepia.ceph.com'
2026-02-24T09:29:27.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T09:29:27.558 DEBUG:teuthology.orchestra.run.trial036:> true
2026-02-24T09:29:27.647 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial036.front.sepia.ceph.com'
2026-02-24T09:29:27.647 DEBUG:teuthology.orchestra.run.trial036:> uname -m
2026-02-24T09:29:27.706 INFO:teuthology.orchestra.run.trial036.stdout:x86_64
2026-02-24T09:29:27.707 DEBUG:teuthology.orchestra.run.trial036:> cat /etc/os-release
2026-02-24T09:29:27.766 INFO:teuthology.orchestra.run.trial036.stdout:NAME="CentOS Stream"
2026-02-24T09:29:27.767 INFO:teuthology.orchestra.run.trial036.stdout:VERSION="9"
2026-02-24T09:29:27.767 INFO:teuthology.orchestra.run.trial036.stdout:ID="centos"
2026-02-24T09:29:27.767 INFO:teuthology.orchestra.run.trial036.stdout:ID_LIKE="rhel fedora"
2026-02-24T09:29:27.767 INFO:teuthology.orchestra.run.trial036.stdout:VERSION_ID="9"
2026-02-24T09:29:27.767 INFO:teuthology.orchestra.run.trial036.stdout:PLATFORM_ID="platform:el9"
2026-02-24T09:29:27.767 INFO:teuthology.orchestra.run.trial036.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T09:29:27.768 INFO:teuthology.orchestra.run.trial036.stdout:ANSI_COLOR="0;31"
2026-02-24T09:29:27.768 INFO:teuthology.orchestra.run.trial036.stdout:LOGO="fedora-logo-icon"
2026-02-24T09:29:27.768 INFO:teuthology.orchestra.run.trial036.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T09:29:27.768 INFO:teuthology.orchestra.run.trial036.stdout:HOME_URL="https://centos.org/"
2026-02-24T09:29:27.768 INFO:teuthology.orchestra.run.trial036.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T09:29:27.768 INFO:teuthology.orchestra.run.trial036.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T09:29:27.769 INFO:teuthology.orchestra.run.trial036.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T09:29:27.770 INFO:teuthology.lock.ops:Updating trial036.front.sepia.ceph.com on lock server
2026-02-24T09:29:27.885 INFO:teuthology.lock.ops:Updating [trial158.front.sepia.ceph.com]: set os type and version on server
2026-02-24T09:29:27.885 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial158.front.sepia.ceph.com'
2026-02-24T09:29:27.885 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T09:29:27.956 DEBUG:teuthology.orchestra.run.trial158:> true
2026-02-24T09:29:28.036 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial158.front.sepia.ceph.com'
2026-02-24T09:29:28.036 DEBUG:teuthology.orchestra.run.trial158:> uname -m
2026-02-24T09:29:28.092 INFO:teuthology.orchestra.run.trial158.stdout:x86_64
2026-02-24T09:29:28.093 DEBUG:teuthology.orchestra.run.trial158:> cat /etc/os-release
2026-02-24T09:29:28.149 INFO:teuthology.orchestra.run.trial158.stdout:NAME="CentOS Stream"
2026-02-24T09:29:28.149 INFO:teuthology.orchestra.run.trial158.stdout:VERSION="9"
2026-02-24T09:29:28.149 INFO:teuthology.orchestra.run.trial158.stdout:ID="centos"
2026-02-24T09:29:28.149 INFO:teuthology.orchestra.run.trial158.stdout:ID_LIKE="rhel fedora"
2026-02-24T09:29:28.149 INFO:teuthology.orchestra.run.trial158.stdout:VERSION_ID="9"
2026-02-24T09:29:28.149 INFO:teuthology.orchestra.run.trial158.stdout:PLATFORM_ID="platform:el9"
2026-02-24T09:29:28.150 INFO:teuthology.orchestra.run.trial158.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T09:29:28.150 INFO:teuthology.orchestra.run.trial158.stdout:ANSI_COLOR="0;31"
2026-02-24T09:29:28.150 INFO:teuthology.orchestra.run.trial158.stdout:LOGO="fedora-logo-icon"
2026-02-24T09:29:28.150 INFO:teuthology.orchestra.run.trial158.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T09:29:28.150 INFO:teuthology.orchestra.run.trial158.stdout:HOME_URL="https://centos.org/"
2026-02-24T09:29:28.150 INFO:teuthology.orchestra.run.trial158.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T09:29:28.150 INFO:teuthology.orchestra.run.trial158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T09:29:28.150 INFO:teuthology.orchestra.run.trial158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T09:29:28.151 INFO:teuthology.lock.ops:Updating trial158.front.sepia.ceph.com on lock server
2026-02-24T09:29:28.253 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:29:28.364 INFO:teuthology.dispatcher.supervisor:Running job 67127
2026-02-24T09:29:28.364 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/bin/teuthology -v --owner scheduled_vshankar@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67127 --name vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} -- /home/teuthworker/mnt/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67127/orig.config.yaml
2026-02-24T09:29:28.364 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67127
2026-02-24T09:29:28.364 INFO:teuthology.dispatcher.supervisor:Job PID: 3575131
2026-02-24T09:29:28.364 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-24T09:31:28.365 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:33:28.486 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:35:28.599 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:37:28.717 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:39:28.837 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:41:28.975 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:43:29.096 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:45:29.221 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:47:29.309 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:49:29.432 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:51:29.560 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:53:29.681 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:55:29.753 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:57:29.874 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:57:30.000 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-24T09:57:30.173 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-24T09:57:30.856 INFO:teuthology.orchestra.console.trial158:Power off
2026-02-24T09:57:30.856 DEBUG:teuthology.orchestra.console.trial158:pexpect command: ipmitool -H trial158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T09:57:30.915 INFO:teuthology.orchestra.console.trial036:Power off
2026-02-24T09:57:30.915 DEBUG:teuthology.orchestra.console.trial036:pexpect command: ipmitool -H trial036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T09:57:30.920 DEBUG:teuthology.orchestra.console.trial158:power off output: Chassis Power Control: Down/Off
2026-02-24T09:57:30.933 INFO:teuthology.orchestra.console.trial020:Power off
2026-02-24T09:57:30.934 DEBUG:teuthology.orchestra.console.trial020:pexpect command: ipmitool -H trial020.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T09:57:30.952 DEBUG:teuthology.orchestra.console.trial036:power off output: Chassis Power Control: Down/Off
2026-02-24T09:57:30.970 DEBUG:teuthology.orchestra.console.trial020:power off output: Chassis Power Control: Down/Off
2026-02-24T09:57:34.921 DEBUG:teuthology.orchestra.console.trial158:pexpect command: ipmitool -H trial158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T09:57:34.954 DEBUG:teuthology.orchestra.console.trial036:pexpect command: ipmitool -H trial036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T09:57:34.970 DEBUG:teuthology.orchestra.console.trial020:pexpect command: ipmitool -H trial020.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T09:57:35.032 DEBUG:teuthology.orchestra.console.trial158:check power output: Chassis Power is on
2026-02-24T09:57:35.062 DEBUG:teuthology.orchestra.console.trial036:check power output: Chassis Power is on
2026-02-24T09:57:35.079 DEBUG:teuthology.orchestra.console.trial020:check power output: Chassis Power is on
2026-02-24T09:57:39.034 DEBUG:teuthology.orchestra.console.trial158:pexpect command: ipmitool -H trial158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T09:57:39.063 DEBUG:teuthology.orchestra.console.trial036:pexpect command: ipmitool -H trial036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T09:57:39.079 DEBUG:teuthology.orchestra.console.trial020:pexpect command: ipmitool -H trial020.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T09:57:39.141 DEBUG:teuthology.orchestra.console.trial158:check power output: Chassis Power is off
2026-02-24T09:57:39.141 INFO:teuthology.orchestra.console.trial158:Power off completed
2026-02-24T09:57:39.171 DEBUG:teuthology.orchestra.console.trial036:check power output: Chassis Power is off
2026-02-24T09:57:39.171 INFO:teuthology.orchestra.console.trial036:Power off completed
2026-02-24T09:57:39.188 DEBUG:teuthology.orchestra.console.trial020:check power output: Chassis Power is off
2026-02-24T09:57:39.188 INFO:teuthology.orchestra.console.trial020:Power off completed
2026-02-24T09:57:39.432 INFO:teuthology.lock.ops:unlocked: trial158.front.sepia.ceph.com
2026-02-24T09:57:39.441 INFO:teuthology.lock.ops:unlocked: trial036.front.sepia.ceph.com
2026-02-24T09:57:39.445 INFO:teuthology.lock.ops:unlocked: trial020.front.sepia.ceph.com