2026-02-23T04:38:49.725 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-23T04:38:49.725 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T04:38:49.796 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T04:38:50.047 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x729821617320>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x729820e8d260>, signals=[15])
2026-02-23T04:38:50.047 INFO:teuthology.lock.ops:Start node 'trial045.front.sepia.ceph.com' reimaging
2026-02-23T04:38:50.047 INFO:teuthology.lock.ops:Updating [trial045.front.sepia.ceph.com]: reset os type and version on server
2026-02-23T04:38:50.047 INFO:teuthology.lock.ops:Updating trial045.front.sepia.ceph.com on lock server
2026-02-23T04:38:50.104 INFO:teuthology.lock.ops:Start node 'trial096.front.sepia.ceph.com' reimaging
2026-02-23T04:38:50.104 INFO:teuthology.lock.ops:Updating [trial096.front.sepia.ceph.com]: reset os type and version on server
2026-02-23T04:38:50.104 INFO:teuthology.lock.ops:Updating trial096.front.sepia.ceph.com on lock server
2026-02-23T04:38:50.162 INFO:teuthology.lock.ops:Start node 'trial107.front.sepia.ceph.com' reimaging
2026-02-23T04:38:50.162 INFO:teuthology.lock.ops:Updating [trial107.front.sepia.ceph.com]: reset os type and version on server
2026-02-23T04:38:50.162 INFO:teuthology.lock.ops:Updating trial107.front.sepia.ceph.com on lock server
2026-02-23T04:38:50.393 DEBUG:teuthology.provision.fog.trial045:Requesting image trial_centos_9.stream (ID 3)
2026-02-23T04:38:50.393 DEBUG:teuthology.provision.fog.trial096:Requesting image trial_centos_9.stream (ID 3)
2026-02-23T04:38:50.416 INFO:teuthology.provision.fog.trial045:Scheduling deploy of centos 9.stream
2026-02-23T04:38:50.442 INFO:teuthology.provision.fog.trial096:Scheduling deploy of centos 9.stream
2026-02-23T04:38:50.451 DEBUG:teuthology.provision.fog.trial107:Requesting image trial_centos_9.stream (ID 3)
2026-02-23T04:38:50.480 INFO:teuthology.provision.fog.trial107:Scheduling deploy of centos 9.stream
2026-02-23T04:38:51.855 INFO:teuthology.orchestra.console.trial045:Power off
2026-02-23T04:38:51.855 DEBUG:teuthology.orchestra.console.trial045:pexpect command: ipmitool -H trial045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-23T04:38:51.868 INFO:teuthology.orchestra.console.trial107:Power off
2026-02-23T04:38:51.868 DEBUG:teuthology.orchestra.console.trial107:pexpect command: ipmitool -H trial107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-23T04:38:51.876 INFO:teuthology.orchestra.console.trial096:Power off
2026-02-23T04:38:51.876 DEBUG:teuthology.orchestra.console.trial096:pexpect command: ipmitool -H trial096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-23T04:38:51.894 DEBUG:teuthology.orchestra.console.trial045:power off output: Chassis Power Control: Down/Off
2026-02-23T04:38:51.906 DEBUG:teuthology.orchestra.console.trial107:power off output: Chassis Power Control: Down/Off
2026-02-23T04:38:51.914 DEBUG:teuthology.orchestra.console.trial096:power off output: Chassis Power Control: Down/Off
2026-02-23T04:38:55.899 DEBUG:teuthology.orchestra.console.trial045:pexpect command: ipmitool -H trial045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T04:38:55.907 DEBUG:teuthology.orchestra.console.trial107:pexpect command: ipmitool -H trial107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T04:38:55.914 DEBUG:teuthology.orchestra.console.trial096:pexpect command: ipmitool -H trial096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T04:38:56.009 DEBUG:teuthology.orchestra.console.trial045:check power output: Chassis Power is off
2026-02-23T04:38:56.009 INFO:teuthology.orchestra.console.trial045:Power off completed
2026-02-23T04:38:56.015 DEBUG:teuthology.orchestra.console.trial107:check power output: Chassis Power is off
2026-02-23T04:38:56.015 INFO:teuthology.orchestra.console.trial107:Power off completed
2026-02-23T04:38:56.023 DEBUG:teuthology.orchestra.console.trial096:check power output: Chassis Power is off
2026-02-23T04:38:56.023 INFO:teuthology.orchestra.console.trial096:Power off completed
2026-02-23T04:38:56.110 INFO:teuthology.orchestra.console.trial045:Power on
2026-02-23T04:38:56.110 DEBUG:teuthology.orchestra.console.trial045:pexpect command: ipmitool -H trial045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-23T04:38:56.118 INFO:teuthology.orchestra.console.trial107:Power on
2026-02-23T04:38:56.119 DEBUG:teuthology.orchestra.console.trial107:pexpect command: ipmitool -H trial107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-23T04:38:56.126 INFO:teuthology.orchestra.console.trial096:Power on
2026-02-23T04:38:56.126 DEBUG:teuthology.orchestra.console.trial096:pexpect command: ipmitool -H trial096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-23T04:38:56.151 DEBUG:teuthology.orchestra.console.trial045:power on output: Chassis Power Control: Up/On
2026-02-23T04:38:56.158 DEBUG:teuthology.orchestra.console.trial107:power on output: Chassis Power Control: Up/On
2026-02-23T04:38:56.166 DEBUG:teuthology.orchestra.console.trial096:power on output: Chassis Power Control: Up/On
2026-02-23T04:39:00.155 DEBUG:teuthology.orchestra.console.trial045:pexpect command: ipmitool -H trial045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T04:39:00.164 DEBUG:teuthology.orchestra.console.trial107:pexpect command: ipmitool -H trial107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T04:39:00.172 DEBUG:teuthology.orchestra.console.trial096:pexpect command: ipmitool -H trial096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T04:39:00.267 DEBUG:teuthology.orchestra.console.trial045:check power output: Chassis Power is off
2026-02-23T04:39:00.274 DEBUG:teuthology.orchestra.console.trial107:check power output: Chassis Power is on
2026-02-23T04:39:00.274 INFO:teuthology.orchestra.console.trial107:Power on completed
2026-02-23T04:39:00.283 DEBUG:teuthology.orchestra.console.trial096:check power output: Chassis Power is on
2026-02-23T04:39:00.283 INFO:teuthology.orchestra.console.trial096:Power on completed
2026-02-23T04:39:00.375 INFO:teuthology.provision.fog.trial107:Waiting for deploy to finish
2026-02-23T04:39:00.383 INFO:teuthology.provision.fog.trial096:Waiting for deploy to finish
2026-02-23T04:39:04.271 DEBUG:teuthology.orchestra.console.trial045:pexpect command: ipmitool -H trial045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T04:39:04.382 DEBUG:teuthology.orchestra.console.trial045:check power output: Chassis Power is on
2026-02-23T04:39:04.382 INFO:teuthology.orchestra.console.trial045:Power on completed
2026-02-23T04:39:04.483 INFO:teuthology.provision.fog.trial045:Waiting for deploy to finish
2026-02-23T04:40:48.024 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T04:40:48.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T04:40:48.176 ERROR:teuthology.orchestra.connection:Error authenticating with trial107.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2026-02-23T04:40:51.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T04:40:54.849 WARNING:teuthology.provision.fog.trial045:[Errno None] Unable to connect to port 22 on 10.20.193.45
2026-02-23T04:41:00.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T04:41:03.936 WARNING:teuthology.provision.fog.trial045:[Errno None] Unable to connect to port 22 on 10.20.193.45
2026-02-23T04:41:09.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T04:41:13.024 WARNING:teuthology.provision.fog.trial045:[Errno None] Unable to connect to port 22 on 10.20.193.45
2026-02-23T04:41:19.025 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T04:41:22.112 WARNING:teuthology.provision.fog.trial045:[Errno None] Unable to connect to port 22 on 10.20.193.45
2026-02-23T04:41:28.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T04:41:31.200 WARNING:teuthology.provision.fog.trial045:[Errno None] Unable to connect to port 22 on 10.20.193.45
2026-02-23T04:41:37.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T04:41:48.153 WARNING:teuthology.provision.fog.trial096:timed out
2026-02-23T04:41:52.179 WARNING:teuthology.provision.fog.trial107:timed out
2026-02-23T04:41:54.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T04:41:58.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T04:42:03.451 ERROR:teuthology.orchestra.connection:Error authenticating with trial107.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-23T04:42:05.519 ERROR:teuthology.orchestra.connection:Error authenticating with trial096.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-23T04:42:07.670 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-23T04:42:07.670 DEBUG:teuthology.orchestra.run.trial107:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-23T04:42:09.630 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-23T04:42:09.630 DEBUG:teuthology.orchestra.run.trial096:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-23T04:42:09.957 INFO:teuthology.provision.fog.trial096:Node is ready
2026-02-23T04:42:09.957 DEBUG:teuthology.orchestra.run.trial096:> hostname
2026-02-23T04:42:10.010 INFO:teuthology.orchestra.run.trial096.stdout:trial096.front.sepia.ceph.com
2026-02-23T04:42:10.011 DEBUG:teuthology.orchestra.run.trial096:> grep trial096.front.sepia.ceph.com /etc/hosts
2026-02-23T04:42:10.064 INFO:teuthology.orchestra.run.trial096.stdout:10.20.193.96 trial096.front.sepia.ceph.com trial096
2026-02-23T04:42:10.065 DEBUG:teuthology.orchestra.run.trial096:> sudo hostname trial096
2026-02-23T04:42:10.131 DEBUG:teuthology.orchestra.run.trial096:> sudo sed -i -e 's/trial096.front.sepia.ceph.com/trial096/g' /etc/hosts
2026-02-23T04:42:10.192 DEBUG:teuthology.orchestra.run.trial096:> sudo sed -i -e 's/10.20.193.96/10.20.193.96/g' /etc/hosts
2026-02-23T04:42:10.255 DEBUG:teuthology.orchestra.run.trial096:> sudo sed -i -e 's/trial096.front.sepia.ceph.com/trial096/g' /etc/hostname
2026-02-23T04:42:10.319 DEBUG:teuthology.orchestra.run.trial096:> sudo hostname trial096
2026-02-23T04:42:10.385 DEBUG:teuthology.orchestra.run.trial096:> cat /etc/os-release
2026-02-23T04:42:10.441 INFO:teuthology.orchestra.run.trial096.stdout:NAME="CentOS Stream"
2026-02-23T04:42:10.442 INFO:teuthology.orchestra.run.trial096.stdout:VERSION="9"
2026-02-23T04:42:10.442 INFO:teuthology.orchestra.run.trial096.stdout:ID="centos"
2026-02-23T04:42:10.442 INFO:teuthology.orchestra.run.trial096.stdout:ID_LIKE="rhel fedora"
2026-02-23T04:42:10.442 INFO:teuthology.orchestra.run.trial096.stdout:VERSION_ID="9"
2026-02-23T04:42:10.442 INFO:teuthology.orchestra.run.trial096.stdout:PLATFORM_ID="platform:el9"
2026-02-23T04:42:10.442 INFO:teuthology.orchestra.run.trial096.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-23T04:42:10.442 INFO:teuthology.orchestra.run.trial096.stdout:ANSI_COLOR="0;31"
2026-02-23T04:42:10.442 INFO:teuthology.orchestra.run.trial096.stdout:LOGO="fedora-logo-icon"
2026-02-23T04:42:10.442 INFO:teuthology.orchestra.run.trial096.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-23T04:42:10.443 INFO:teuthology.orchestra.run.trial096.stdout:HOME_URL="https://centos.org/"
2026-02-23T04:42:10.443 INFO:teuthology.orchestra.run.trial096.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-23T04:42:10.443 INFO:teuthology.orchestra.run.trial096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-23T04:42:10.443 INFO:teuthology.orchestra.run.trial096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-23T04:42:10.444 INFO:teuthology.provision.fog.trial096:Deploy complete!
2026-02-23T04:42:10.445 DEBUG:teuthology.parallel:result is None
2026-02-23T04:42:12.819 INFO:teuthology.provision.fog.trial107:Node is ready
2026-02-23T04:42:12.819 DEBUG:teuthology.orchestra.run.trial107:> hostname
2026-02-23T04:42:12.842 INFO:teuthology.orchestra.run.trial107.stdout:trial107.front.sepia.ceph.com
2026-02-23T04:42:12.842 DEBUG:teuthology.orchestra.run.trial107:> grep trial107.front.sepia.ceph.com /etc/hosts
2026-02-23T04:42:12.927 INFO:teuthology.orchestra.run.trial107.stdout:10.20.193.107 trial107.front.sepia.ceph.com trial107
2026-02-23T04:42:12.928 DEBUG:teuthology.orchestra.run.trial107:> sudo hostname trial107
2026-02-23T04:42:13.066 DEBUG:teuthology.orchestra.run.trial107:> sudo sed -i -e 's/trial107.front.sepia.ceph.com/trial107/g' /etc/hosts
2026-02-23T04:42:13.093 DEBUG:teuthology.orchestra.run.trial107:> sudo sed -i -e 's/10.20.193.107/10.20.193.107/g' /etc/hosts
2026-02-23T04:42:13.161 DEBUG:teuthology.orchestra.run.trial107:> sudo sed -i -e 's/trial107.front.sepia.ceph.com/trial107/g' /etc/hostname
2026-02-23T04:42:13.227 DEBUG:teuthology.orchestra.run.trial107:> sudo hostname trial107
2026-02-23T04:42:13.366 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-23T04:42:13.366 DEBUG:teuthology.orchestra.run.trial045:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-23T04:42:13.372 DEBUG:teuthology.orchestra.run.trial107:> cat /etc/os-release
2026-02-23T04:42:13.429 INFO:teuthology.orchestra.run.trial107.stdout:NAME="CentOS Stream"
2026-02-23T04:42:13.429 INFO:teuthology.orchestra.run.trial107.stdout:VERSION="9"
2026-02-23T04:42:13.429 INFO:teuthology.orchestra.run.trial107.stdout:ID="centos"
2026-02-23T04:42:13.429 INFO:teuthology.orchestra.run.trial107.stdout:ID_LIKE="rhel fedora"
2026-02-23T04:42:13.429 INFO:teuthology.orchestra.run.trial107.stdout:VERSION_ID="9"
2026-02-23T04:42:13.429 INFO:teuthology.orchestra.run.trial107.stdout:PLATFORM_ID="platform:el9"
2026-02-23T04:42:13.429 INFO:teuthology.orchestra.run.trial107.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-23T04:42:13.429 INFO:teuthology.orchestra.run.trial107.stdout:ANSI_COLOR="0;31"
2026-02-23T04:42:13.429 INFO:teuthology.orchestra.run.trial107.stdout:LOGO="fedora-logo-icon"
2026-02-23T04:42:13.429 INFO:teuthology.orchestra.run.trial107.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-23T04:42:13.429 INFO:teuthology.orchestra.run.trial107.stdout:HOME_URL="https://centos.org/"
2026-02-23T04:42:13.429 INFO:teuthology.orchestra.run.trial107.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-23T04:42:13.429 INFO:teuthology.orchestra.run.trial107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-23T04:42:13.429 INFO:teuthology.orchestra.run.trial107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-23T04:42:13.429 INFO:teuthology.provision.fog.trial107:Deploy complete!
2026-02-23T04:42:13.430 DEBUG:teuthology.parallel:result is None
2026-02-23T04:42:13.517 INFO:teuthology.provision.fog.trial045:Node is ready
2026-02-23T04:42:13.517 DEBUG:teuthology.orchestra.run.trial045:> hostname
2026-02-23T04:42:13.569 INFO:teuthology.orchestra.run.trial045.stdout:trial045.front.sepia.ceph.com
2026-02-23T04:42:13.569 DEBUG:teuthology.orchestra.run.trial045:> grep trial045.front.sepia.ceph.com /etc/hosts
2026-02-23T04:42:13.623 INFO:teuthology.orchestra.run.trial045.stdout:10.20.193.45 trial045.front.sepia.ceph.com trial045
2026-02-23T04:42:13.624 DEBUG:teuthology.orchestra.run.trial045:> sudo hostname trial045
2026-02-23T04:42:13.693 DEBUG:teuthology.orchestra.run.trial045:> sudo sed -i -e 's/trial045.front.sepia.ceph.com/trial045/g' /etc/hosts
2026-02-23T04:42:13.757 DEBUG:teuthology.orchestra.run.trial045:> sudo sed -i -e 's/10.20.193.45/10.20.193.45/g' /etc/hosts
2026-02-23T04:42:13.820 DEBUG:teuthology.orchestra.run.trial045:> sudo sed -i -e 's/trial045.front.sepia.ceph.com/trial045/g' /etc/hostname
2026-02-23T04:42:13.881 DEBUG:teuthology.orchestra.run.trial045:> sudo hostname trial045
2026-02-23T04:42:13.943 DEBUG:teuthology.orchestra.run.trial045:> cat /etc/os-release
2026-02-23T04:42:13.998 INFO:teuthology.orchestra.run.trial045.stdout:NAME="CentOS Stream"
2026-02-23T04:42:13.999 INFO:teuthology.orchestra.run.trial045.stdout:VERSION="9"
2026-02-23T04:42:13.999 INFO:teuthology.orchestra.run.trial045.stdout:ID="centos"
2026-02-23T04:42:13.999 INFO:teuthology.orchestra.run.trial045.stdout:ID_LIKE="rhel fedora"
2026-02-23T04:42:13.999 INFO:teuthology.orchestra.run.trial045.stdout:VERSION_ID="9"
2026-02-23T04:42:13.999 INFO:teuthology.orchestra.run.trial045.stdout:PLATFORM_ID="platform:el9"
2026-02-23T04:42:13.999 INFO:teuthology.orchestra.run.trial045.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-23T04:42:13.999 INFO:teuthology.orchestra.run.trial045.stdout:ANSI_COLOR="0;31"
2026-02-23T04:42:13.999 INFO:teuthology.orchestra.run.trial045.stdout:LOGO="fedora-logo-icon"
2026-02-23T04:42:14.000 INFO:teuthology.orchestra.run.trial045.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-23T04:42:14.000 INFO:teuthology.orchestra.run.trial045.stdout:HOME_URL="https://centos.org/"
2026-02-23T04:42:14.000 INFO:teuthology.orchestra.run.trial045.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-23T04:42:14.000 INFO:teuthology.orchestra.run.trial045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-23T04:42:14.000 INFO:teuthology.orchestra.run.trial045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-23T04:42:14.000 INFO:teuthology.provision.fog.trial045:Deploy complete!
2026-02-23T04:42:14.001 DEBUG:teuthology.parallel:result is None
2026-02-23T04:42:14.524 INFO:teuthology.lock.ops:Checking trial045.front.sepia.ceph.com
2026-02-23T04:42:14.524 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-23T04:42:14.625 INFO:teuthology.lock.ops:Checking trial096.front.sepia.ceph.com
2026-02-23T04:42:14.625 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-23T04:42:14.688 INFO:teuthology.lock.ops:Checking trial107.front.sepia.ceph.com
2026-02-23T04:42:14.688 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-23T04:42:14.748 INFO:teuthology.lock.ops:Updating [trial045.front.sepia.ceph.com]: set os type and version on server
2026-02-23T04:42:14.748 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial045.front.sepia.ceph.com'
2026-02-23T04:42:14.749 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T04:42:14.809 DEBUG:teuthology.orchestra.run.trial045:> true
2026-02-23T04:42:14.880 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial045.front.sepia.ceph.com'
2026-02-23T04:42:14.880 DEBUG:teuthology.orchestra.run.trial045:> uname -m
2026-02-23T04:42:14.935 INFO:teuthology.orchestra.run.trial045.stdout:x86_64
2026-02-23T04:42:14.936 DEBUG:teuthology.orchestra.run.trial045:> cat /etc/os-release
2026-02-23T04:42:14.990 INFO:teuthology.orchestra.run.trial045.stdout:NAME="CentOS Stream"
2026-02-23T04:42:14.990 INFO:teuthology.orchestra.run.trial045.stdout:VERSION="9"
2026-02-23T04:42:14.990 INFO:teuthology.orchestra.run.trial045.stdout:ID="centos"
2026-02-23T04:42:14.990 INFO:teuthology.orchestra.run.trial045.stdout:ID_LIKE="rhel fedora"
2026-02-23T04:42:14.990 INFO:teuthology.orchestra.run.trial045.stdout:VERSION_ID="9"
2026-02-23T04:42:14.991 INFO:teuthology.orchestra.run.trial045.stdout:PLATFORM_ID="platform:el9"
2026-02-23T04:42:14.991 INFO:teuthology.orchestra.run.trial045.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-23T04:42:14.991 INFO:teuthology.orchestra.run.trial045.stdout:ANSI_COLOR="0;31"
2026-02-23T04:42:14.991 INFO:teuthology.orchestra.run.trial045.stdout:LOGO="fedora-logo-icon"
2026-02-23T04:42:14.991 INFO:teuthology.orchestra.run.trial045.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-23T04:42:14.991 INFO:teuthology.orchestra.run.trial045.stdout:HOME_URL="https://centos.org/"
2026-02-23T04:42:14.991 INFO:teuthology.orchestra.run.trial045.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-23T04:42:14.991 INFO:teuthology.orchestra.run.trial045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-23T04:42:14.991 INFO:teuthology.orchestra.run.trial045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-23T04:42:14.992 INFO:teuthology.lock.ops:Updating trial045.front.sepia.ceph.com on lock server
2026-02-23T04:42:15.149 INFO:teuthology.lock.ops:Updating [trial096.front.sepia.ceph.com]: set os type and version on server
2026-02-23T04:42:15.149 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial096.front.sepia.ceph.com'
2026-02-23T04:42:15.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T04:42:15.219 DEBUG:teuthology.orchestra.run.trial096:> true
2026-02-23T04:42:15.295 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial096.front.sepia.ceph.com'
2026-02-23T04:42:15.295 DEBUG:teuthology.orchestra.run.trial096:> uname -m
2026-02-23T04:42:15.351 INFO:teuthology.orchestra.run.trial096.stdout:x86_64
2026-02-23T04:42:15.351 DEBUG:teuthology.orchestra.run.trial096:> cat /etc/os-release
2026-02-23T04:42:15.407 INFO:teuthology.orchestra.run.trial096.stdout:NAME="CentOS Stream"
2026-02-23T04:42:15.407 INFO:teuthology.orchestra.run.trial096.stdout:VERSION="9"
2026-02-23T04:42:15.407 INFO:teuthology.orchestra.run.trial096.stdout:ID="centos"
2026-02-23T04:42:15.407 INFO:teuthology.orchestra.run.trial096.stdout:ID_LIKE="rhel fedora"
2026-02-23T04:42:15.407 INFO:teuthology.orchestra.run.trial096.stdout:VERSION_ID="9"
2026-02-23T04:42:15.407 INFO:teuthology.orchestra.run.trial096.stdout:PLATFORM_ID="platform:el9"
2026-02-23T04:42:15.407 INFO:teuthology.orchestra.run.trial096.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-23T04:42:15.408 INFO:teuthology.orchestra.run.trial096.stdout:ANSI_COLOR="0;31"
2026-02-23T04:42:15.408 INFO:teuthology.orchestra.run.trial096.stdout:LOGO="fedora-logo-icon"
2026-02-23T04:42:15.408 INFO:teuthology.orchestra.run.trial096.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-23T04:42:15.408 INFO:teuthology.orchestra.run.trial096.stdout:HOME_URL="https://centos.org/"
2026-02-23T04:42:15.408 INFO:teuthology.orchestra.run.trial096.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-23T04:42:15.408 INFO:teuthology.orchestra.run.trial096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-23T04:42:15.408 INFO:teuthology.orchestra.run.trial096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-23T04:42:15.409 INFO:teuthology.lock.ops:Updating trial096.front.sepia.ceph.com on lock server
2026-02-23T04:42:15.513 INFO:teuthology.lock.ops:Updating [trial107.front.sepia.ceph.com]: set os type and version on server
2026-02-23T04:42:15.513 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial107.front.sepia.ceph.com'
2026-02-23T04:42:15.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T04:42:15.585 DEBUG:teuthology.orchestra.run.trial107:> true
2026-02-23T04:42:15.664 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial107.front.sepia.ceph.com'
2026-02-23T04:42:15.664 DEBUG:teuthology.orchestra.run.trial107:> uname -m
2026-02-23T04:42:15.720 INFO:teuthology.orchestra.run.trial107.stdout:x86_64
2026-02-23T04:42:15.721 DEBUG:teuthology.orchestra.run.trial107:> cat /etc/os-release
2026-02-23T04:42:15.777 INFO:teuthology.orchestra.run.trial107.stdout:NAME="CentOS Stream"
2026-02-23T04:42:15.777 INFO:teuthology.orchestra.run.trial107.stdout:VERSION="9"
2026-02-23T04:42:15.777 INFO:teuthology.orchestra.run.trial107.stdout:ID="centos"
2026-02-23T04:42:15.777 INFO:teuthology.orchestra.run.trial107.stdout:ID_LIKE="rhel fedora"
2026-02-23T04:42:15.777 INFO:teuthology.orchestra.run.trial107.stdout:VERSION_ID="9"
2026-02-23T04:42:15.777 INFO:teuthology.orchestra.run.trial107.stdout:PLATFORM_ID="platform:el9"
2026-02-23T04:42:15.778 INFO:teuthology.orchestra.run.trial107.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-23T04:42:15.778 INFO:teuthology.orchestra.run.trial107.stdout:ANSI_COLOR="0;31"
2026-02-23T04:42:15.778 INFO:teuthology.orchestra.run.trial107.stdout:LOGO="fedora-logo-icon"
2026-02-23T04:42:15.778 INFO:teuthology.orchestra.run.trial107.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-23T04:42:15.778 INFO:teuthology.orchestra.run.trial107.stdout:HOME_URL="https://centos.org/"
2026-02-23T04:42:15.778 INFO:teuthology.orchestra.run.trial107.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-23T04:42:15.778 INFO:teuthology.orchestra.run.trial107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-23T04:42:15.778 INFO:teuthology.orchestra.run.trial107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-23T04:42:15.779 INFO:teuthology.lock.ops:Updating trial107.front.sepia.ceph.com on lock server
2026-02-23T04:42:15.880 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T04:42:15.975 INFO:teuthology.dispatcher.supervisor:Running job 64831
2026-02-23T04:42:15.975 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/bin/teuthology -v --owner scheduled_vshankar@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/vshankar-2026-02-23_04:04:18-fs-wip-vshankar-testing-20260222.101816-testing-default-trial/64831 --name vshankar-2026-02-23_04:04:18-fs-wip-vshankar-testing-20260222.101816-testing-default-trial --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} -- /home/teuthworker/mnt/teuthology/vshankar-2026-02-23_04:04:18-fs-wip-vshankar-testing-20260222.101816-testing-default-trial/64831/orig.config.yaml
2026-02-23T04:42:15.975 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/vshankar-2026-02-23_04:04:18-fs-wip-vshankar-testing-20260222.101816-testing-default-trial/64831
2026-02-23T04:42:15.975 INFO:teuthology.dispatcher.supervisor:Job PID: 1658811
2026-02-23T04:42:15.975 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-23T04:44:15.976 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T04:46:16.099 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T04:48:16.229 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T04:50:16.302 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T04:52:16.425 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T04:54:16.550 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T04:56:16.670 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T04:58:16.799 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T05:00:16.922 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T05:02:17.001 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T05:04:17.117 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T05:04:17.234 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-23T05:04:17.414 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-23T05:04:18.172 INFO:teuthology.orchestra.console.trial107:Power off
2026-02-23T05:04:18.172 DEBUG:teuthology.orchestra.console.trial107:pexpect command: ipmitool -H trial107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-23T05:04:18.185 INFO:teuthology.orchestra.console.trial045:Power off
2026-02-23T05:04:18.185 DEBUG:teuthology.orchestra.console.trial045:pexpect command: ipmitool -H trial045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-23T05:04:18.193 INFO:teuthology.orchestra.console.trial096:Power off
2026-02-23T05:04:18.193 DEBUG:teuthology.orchestra.console.trial096:pexpect command: ipmitool -H trial096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-23T05:04:18.212 DEBUG:teuthology.orchestra.console.trial107:power off output: Chassis Power Control: Down/Off
2026-02-23T05:04:18.223 DEBUG:teuthology.orchestra.console.trial045:power off output: Chassis Power Control: Down/Off
2026-02-23T05:04:18.231 DEBUG:teuthology.orchestra.console.trial096:power off output: Chassis Power Control: Down/Off
2026-02-23T05:04:22.212 DEBUG:teuthology.orchestra.console.trial107:pexpect command: ipmitool -H trial107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T05:04:22.224 DEBUG:teuthology.orchestra.console.trial045:pexpect command: ipmitool -H trial045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T05:04:22.232 DEBUG:teuthology.orchestra.console.trial096:pexpect command: ipmitool -H trial096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T05:04:22.321 DEBUG:teuthology.orchestra.console.trial107:check power output: Chassis Power is on
2026-02-23T05:04:22.332 DEBUG:teuthology.orchestra.console.trial045:check power output: Chassis Power is on
2026-02-23T05:04:22.342 DEBUG:teuthology.orchestra.console.trial096:check power output: Chassis Power is on
2026-02-23T05:04:26.322 DEBUG:teuthology.orchestra.console.trial107:pexpect command: ipmitool -H trial107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T05:04:26.333 DEBUG:teuthology.orchestra.console.trial045:pexpect command: ipmitool -H trial045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T05:04:26.342 DEBUG:teuthology.orchestra.console.trial096:pexpect command: ipmitool -H trial096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T05:04:26.431 DEBUG:teuthology.orchestra.console.trial107:check power output: Chassis Power is off
2026-02-23T05:04:26.431 INFO:teuthology.orchestra.console.trial107:Power off completed
2026-02-23T05:04:26.440 DEBUG:teuthology.orchestra.console.trial045:check power output: Chassis Power is off
2026-02-23T05:04:26.440 INFO:teuthology.orchestra.console.trial045:Power off completed
2026-02-23T05:04:26.450 DEBUG:teuthology.orchestra.console.trial096:check power output: Chassis Power is off
2026-02-23T05:04:26.450 INFO:teuthology.orchestra.console.trial096:Power off completed
2026-02-23T05:04:26.739 INFO:teuthology.lock.ops:unlocked: trial107.front.sepia.ceph.com
2026-02-23T05:04:26.749 INFO:teuthology.lock.ops:unlocked: trial045.front.sepia.ceph.com
2026-02-23T05:04:26.753 INFO:teuthology.lock.ops:unlocked: trial096.front.sepia.ceph.com