2026-02-12T22:44:58.492 INFO:root:teuthology version: 1.2.4.dev7+g3e9168473
2026-02-12T22:44:58.492 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T22:44:58.564 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T22:44:58.816 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x71fa19c0c290>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x71fa1940c7c0>, signals=[15])
2026-02-12T22:44:58.816 INFO:teuthology.lock.ops:Start node 'trial048.front.sepia.ceph.com' reimaging
2026-02-12T22:44:58.816 INFO:teuthology.lock.ops:Updating [trial048.front.sepia.ceph.com]: reset os type and version on server
2026-02-12T22:44:58.817 INFO:teuthology.lock.ops:Updating trial048.front.sepia.ceph.com on lock server
2026-02-12T22:44:58.872 INFO:teuthology.lock.ops:Start node 'trial110.front.sepia.ceph.com' reimaging
2026-02-12T22:44:58.872 INFO:teuthology.lock.ops:Updating [trial110.front.sepia.ceph.com]: reset os type and version on server
2026-02-12T22:44:58.872 INFO:teuthology.lock.ops:Updating trial110.front.sepia.ceph.com on lock server
2026-02-12T22:44:58.929 INFO:teuthology.lock.ops:Start node 'trial156.front.sepia.ceph.com' reimaging
2026-02-12T22:44:58.929 INFO:teuthology.lock.ops:Updating [trial156.front.sepia.ceph.com]: reset os type and version on server
2026-02-12T22:44:58.929 INFO:teuthology.lock.ops:Updating trial156.front.sepia.ceph.com on lock server
2026-02-12T22:44:59.159 DEBUG:teuthology.provision.fog.trial048:Requesting image trial_centos_9.stream (ID 3)
2026-02-12T22:44:59.159 DEBUG:teuthology.provision.fog.trial110:Requesting image trial_centos_9.stream (ID 3)
2026-02-12T22:44:59.211 INFO:teuthology.provision.fog.trial110:Scheduling deploy of centos 9.stream
2026-02-12T22:44:59.213 INFO:teuthology.provision.fog.trial048:Scheduling deploy of centos 9.stream
2026-02-12T22:44:59.214 DEBUG:teuthology.provision.fog.trial156:Requesting image trial_centos_9.stream (ID 3)
2026-02-12T22:44:59.248 INFO:teuthology.provision.fog.trial156:Scheduling deploy of centos 9.stream
2026-02-12T22:45:00.785 INFO:teuthology.orchestra.console.trial110:Power off
2026-02-12T22:45:00.785 DEBUG:teuthology.orchestra.console.trial110:pexpect command: ipmitool -H trial110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T22:45:00.840 INFO:teuthology.orchestra.console.trial048:Power off
2026-02-12T22:45:00.840 DEBUG:teuthology.orchestra.console.trial048:pexpect command: ipmitool -H trial048.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T22:45:00.846 DEBUG:teuthology.orchestra.console.trial110:power off output: Chassis Power Control: Down/Off
2026-02-12T22:45:00.858 INFO:teuthology.orchestra.console.trial156:Power off
2026-02-12T22:45:00.858 DEBUG:teuthology.orchestra.console.trial156:pexpect command: ipmitool -H trial156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T22:45:00.876 DEBUG:teuthology.orchestra.console.trial048:power off output: Chassis Power Control: Down/Off
2026-02-12T22:45:00.896 DEBUG:teuthology.orchestra.console.trial156:power off output: Chassis Power Control: Down/Off
2026-02-12T22:45:04.851 DEBUG:teuthology.orchestra.console.trial110:pexpect command: ipmitool -H trial110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T22:45:04.877 DEBUG:teuthology.orchestra.console.trial048:pexpect command: ipmitool -H trial048.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T22:45:04.897 DEBUG:teuthology.orchestra.console.trial156:pexpect command: ipmitool -H trial156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T22:45:04.959 DEBUG:teuthology.orchestra.console.trial110:check power output: Chassis Power is off
2026-02-12T22:45:04.960 INFO:teuthology.orchestra.console.trial110:Power off completed
2026-02-12T22:45:04.986 DEBUG:teuthology.orchestra.console.trial048:check power output: Chassis Power is off
2026-02-12T22:45:04.986 INFO:teuthology.orchestra.console.trial048:Power off completed
2026-02-12T22:45:05.004 DEBUG:teuthology.orchestra.console.trial156:check power output: Chassis Power is off
2026-02-12T22:45:05.005 INFO:teuthology.orchestra.console.trial156:Power off completed
2026-02-12T22:45:05.061 INFO:teuthology.orchestra.console.trial110:Power on
2026-02-12T22:45:05.061 DEBUG:teuthology.orchestra.console.trial110:pexpect command: ipmitool -H trial110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-12T22:45:05.087 INFO:teuthology.orchestra.console.trial048:Power on
2026-02-12T22:45:05.088 DEBUG:teuthology.orchestra.console.trial048:pexpect command: ipmitool -H trial048.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-12T22:45:05.101 DEBUG:teuthology.orchestra.console.trial110:power on output: Chassis Power Control: Up/On
2026-02-12T22:45:05.105 INFO:teuthology.orchestra.console.trial156:Power on
2026-02-12T22:45:05.106 DEBUG:teuthology.orchestra.console.trial156:pexpect command: ipmitool -H trial156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-12T22:45:05.125 DEBUG:teuthology.orchestra.console.trial048:power on output: Chassis Power Control: Up/On
2026-02-12T22:45:05.144 DEBUG:teuthology.orchestra.console.trial156:power on output: Chassis Power Control: Up/On
2026-02-12T22:45:09.106 DEBUG:teuthology.orchestra.console.trial110:pexpect command: ipmitool -H trial110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T22:45:09.126 DEBUG:teuthology.orchestra.console.trial048:pexpect command: ipmitool -H trial048.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T22:45:09.145 DEBUG:teuthology.orchestra.console.trial156:pexpect command: ipmitool -H trial156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T22:45:09.215 DEBUG:teuthology.orchestra.console.trial110:check power output: Chassis Power is on
2026-02-12T22:45:09.216 INFO:teuthology.orchestra.console.trial110:Power on completed
2026-02-12T22:45:09.235 DEBUG:teuthology.orchestra.console.trial048:check power output: Chassis Power is on
2026-02-12T22:45:09.236 INFO:teuthology.orchestra.console.trial048:Power on completed
2026-02-12T22:45:09.254 DEBUG:teuthology.orchestra.console.trial156:check power output: Chassis Power is on
2026-02-12T22:45:09.255 INFO:teuthology.orchestra.console.trial156:Power on completed
2026-02-12T22:45:09.316 INFO:teuthology.provision.fog.trial110:Waiting for deploy to finish
2026-02-12T22:45:09.336 INFO:teuthology.provision.fog.trial048:Waiting for deploy to finish
2026-02-12T22:45:09.356 INFO:teuthology.provision.fog.trial156:Waiting for deploy to finish
2026-02-12T22:46:57.281 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T22:46:57.319 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial048.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T22:47:12.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T22:47:15.549 WARNING:teuthology.provision.fog.trial110:[Errno None] Unable to connect to port 22 on 10.20.193.110
2026-02-12T22:47:21.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T22:47:24.637 WARNING:teuthology.provision.fog.trial110:[Errno None] Unable to connect to port 22 on 10.20.193.110
2026-02-12T22:47:30.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T22:47:33.725 WARNING:teuthology.provision.fog.trial110:[Errno None] Unable to connect to port 22 on 10.20.193.110
2026-02-12T22:47:39.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T22:47:42.813 WARNING:teuthology.provision.fog.trial110:[Errno None] Unable to connect to port 22 on 10.20.193.110
2026-02-12T22:47:48.820 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T22:47:57.293 WARNING:teuthology.provision.fog.trial156:timed out
2026-02-12T22:47:57.321 WARNING:teuthology.provision.fog.trial048:timed out
2026-02-12T22:48:03.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T22:48:03.322 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial048.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T22:48:03.462 ERROR:teuthology.orchestra.connection:Error authenticating with trial156.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-12T22:48:03.482 ERROR:teuthology.orchestra.connection:Error authenticating with trial048.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-12T22:48:07.604 ERROR:teuthology.orchestra.connection:Error authenticating with trial156.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-12T22:48:07.632 ERROR:teuthology.orchestra.connection:Error authenticating with trial048.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-12T22:48:14.742 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-12T22:48:14.743 DEBUG:teuthology.orchestra.run.trial156:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-12T22:48:14.752 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-12T22:48:14.753 DEBUG:teuthology.orchestra.run.trial048:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-12T22:48:14.811 INFO:teuthology.provision.fog.trial156:Node is ready
2026-02-12T22:48:14.811 DEBUG:teuthology.orchestra.run.trial156:> hostname
2026-02-12T22:48:14.865 INFO:teuthology.orchestra.run.trial156.stdout:trial156.front.sepia.ceph.com
2026-02-12T22:48:14.865 DEBUG:teuthology.orchestra.run.trial156:> grep trial156.front.sepia.ceph.com /etc/hosts
2026-02-12T22:48:14.912 INFO:teuthology.provision.fog.trial048:Node is ready
2026-02-12T22:48:14.912 DEBUG:teuthology.orchestra.run.trial048:> hostname
2026-02-12T22:48:14.920 INFO:teuthology.orchestra.run.trial156.stdout:10.20.193.156 trial156.front.sepia.ceph.com trial156
2026-02-12T22:48:14.921 DEBUG:teuthology.orchestra.run.trial156:> sudo hostname trial156
2026-02-12T22:48:14.967 INFO:teuthology.orchestra.run.trial048.stdout:trial048.front.sepia.ceph.com
2026-02-12T22:48:14.968 DEBUG:teuthology.orchestra.run.trial048:> grep trial048.front.sepia.ceph.com /etc/hosts
2026-02-12T22:48:14.988 DEBUG:teuthology.orchestra.run.trial156:> sudo sed -i -e 's/trial156.front.sepia.ceph.com/trial156/g' /etc/hosts
2026-02-12T22:48:15.023 INFO:teuthology.orchestra.run.trial048.stdout:10.20.193.48 trial048.front.sepia.ceph.com trial048
2026-02-12T22:48:15.024 DEBUG:teuthology.orchestra.run.trial048:> sudo hostname trial048
2026-02-12T22:48:15.052 DEBUG:teuthology.orchestra.run.trial156:> sudo sed -i -e 's/10.20.193.156/10.20.193.156/g' /etc/hosts
2026-02-12T22:48:15.092 DEBUG:teuthology.orchestra.run.trial048:> sudo sed -i -e 's/trial048.front.sepia.ceph.com/trial048/g' /etc/hosts
2026-02-12T22:48:15.116 DEBUG:teuthology.orchestra.run.trial156:> sudo sed -i -e 's/trial156.front.sepia.ceph.com/trial156/g' /etc/hostname
2026-02-12T22:48:15.156 DEBUG:teuthology.orchestra.run.trial048:> sudo sed -i -e 's/10.20.193.48/10.20.193.48/g' /etc/hosts
2026-02-12T22:48:15.181 DEBUG:teuthology.orchestra.run.trial156:> sudo hostname trial156
2026-02-12T22:48:15.221 DEBUG:teuthology.orchestra.run.trial048:> sudo sed -i -e 's/trial048.front.sepia.ceph.com/trial048/g' /etc/hostname
2026-02-12T22:48:15.245 DEBUG:teuthology.orchestra.run.trial156:> cat /etc/os-release
2026-02-12T22:48:15.285 DEBUG:teuthology.orchestra.run.trial048:> sudo hostname trial048
2026-02-12T22:48:15.301 INFO:teuthology.orchestra.run.trial156.stdout:NAME="CentOS Stream"
2026-02-12T22:48:15.301 INFO:teuthology.orchestra.run.trial156.stdout:VERSION="9"
2026-02-12T22:48:15.301 INFO:teuthology.orchestra.run.trial156.stdout:ID="centos"
2026-02-12T22:48:15.301 INFO:teuthology.orchestra.run.trial156.stdout:ID_LIKE="rhel fedora"
2026-02-12T22:48:15.301 INFO:teuthology.orchestra.run.trial156.stdout:VERSION_ID="9"
2026-02-12T22:48:15.301 INFO:teuthology.orchestra.run.trial156.stdout:PLATFORM_ID="platform:el9"
2026-02-12T22:48:15.302 INFO:teuthology.orchestra.run.trial156.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T22:48:15.302 INFO:teuthology.orchestra.run.trial156.stdout:ANSI_COLOR="0;31"
2026-02-12T22:48:15.302 INFO:teuthology.orchestra.run.trial156.stdout:LOGO="fedora-logo-icon"
2026-02-12T22:48:15.302 INFO:teuthology.orchestra.run.trial156.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T22:48:15.302 INFO:teuthology.orchestra.run.trial156.stdout:HOME_URL="https://centos.org/"
2026-02-12T22:48:15.302 INFO:teuthology.orchestra.run.trial156.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T22:48:15.302 INFO:teuthology.orchestra.run.trial156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T22:48:15.302 INFO:teuthology.orchestra.run.trial156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T22:48:15.303 INFO:teuthology.provision.fog.trial156:Deploy complete!
2026-02-12T22:48:15.304 DEBUG:teuthology.parallel:result is None
2026-02-12T22:48:15.349 DEBUG:teuthology.orchestra.run.trial048:> cat /etc/os-release
2026-02-12T22:48:15.406 INFO:teuthology.orchestra.run.trial048.stdout:NAME="CentOS Stream"
2026-02-12T22:48:15.406 INFO:teuthology.orchestra.run.trial048.stdout:VERSION="9"
2026-02-12T22:48:15.406 INFO:teuthology.orchestra.run.trial048.stdout:ID="centos"
2026-02-12T22:48:15.406 INFO:teuthology.orchestra.run.trial048.stdout:ID_LIKE="rhel fedora"
2026-02-12T22:48:15.406 INFO:teuthology.orchestra.run.trial048.stdout:VERSION_ID="9"
2026-02-12T22:48:15.406 INFO:teuthology.orchestra.run.trial048.stdout:PLATFORM_ID="platform:el9"
2026-02-12T22:48:15.406 INFO:teuthology.orchestra.run.trial048.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T22:48:15.406 INFO:teuthology.orchestra.run.trial048.stdout:ANSI_COLOR="0;31"
2026-02-12T22:48:15.407 INFO:teuthology.orchestra.run.trial048.stdout:LOGO="fedora-logo-icon"
2026-02-12T22:48:15.407 INFO:teuthology.orchestra.run.trial048.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T22:48:15.407 INFO:teuthology.orchestra.run.trial048.stdout:HOME_URL="https://centos.org/"
2026-02-12T22:48:15.407 INFO:teuthology.orchestra.run.trial048.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T22:48:15.407 INFO:teuthology.orchestra.run.trial048.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T22:48:15.407 INFO:teuthology.orchestra.run.trial048.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T22:48:15.407 INFO:teuthology.provision.fog.trial048:Deploy complete!
2026-02-12T22:48:15.408 DEBUG:teuthology.parallel:result is None
2026-02-12T22:48:24.621 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-12T22:48:24.621 DEBUG:teuthology.orchestra.run.trial110:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-12T22:48:24.787 INFO:teuthology.provision.fog.trial110:Node is ready
2026-02-12T22:48:24.787 DEBUG:teuthology.orchestra.run.trial110:> hostname
2026-02-12T22:48:24.842 INFO:teuthology.orchestra.run.trial110.stdout:trial110.front.sepia.ceph.com
2026-02-12T22:48:24.842 DEBUG:teuthology.orchestra.run.trial110:> grep trial110.front.sepia.ceph.com /etc/hosts
2026-02-12T22:48:24.899 INFO:teuthology.orchestra.run.trial110.stdout:10.20.193.110 trial110.front.sepia.ceph.com trial110
2026-02-12T22:48:24.900 DEBUG:teuthology.orchestra.run.trial110:> sudo hostname trial110
2026-02-12T22:48:24.969 DEBUG:teuthology.orchestra.run.trial110:> sudo sed -i -e 's/trial110.front.sepia.ceph.com/trial110/g' /etc/hosts
2026-02-12T22:48:25.033 DEBUG:teuthology.orchestra.run.trial110:> sudo sed -i -e 's/10.20.193.110/10.20.193.110/g' /etc/hosts
2026-02-12T22:48:25.098 DEBUG:teuthology.orchestra.run.trial110:> sudo sed -i -e 's/trial110.front.sepia.ceph.com/trial110/g' /etc/hostname
2026-02-12T22:48:25.161 DEBUG:teuthology.orchestra.run.trial110:> sudo hostname trial110
2026-02-12T22:48:25.225 DEBUG:teuthology.orchestra.run.trial110:> cat /etc/os-release
2026-02-12T22:48:25.281 INFO:teuthology.orchestra.run.trial110.stdout:NAME="CentOS Stream"
2026-02-12T22:48:25.281 INFO:teuthology.orchestra.run.trial110.stdout:VERSION="9"
2026-02-12T22:48:25.281 INFO:teuthology.orchestra.run.trial110.stdout:ID="centos"
2026-02-12T22:48:25.281 INFO:teuthology.orchestra.run.trial110.stdout:ID_LIKE="rhel fedora"
2026-02-12T22:48:25.282 INFO:teuthology.orchestra.run.trial110.stdout:VERSION_ID="9"
2026-02-12T22:48:25.282 INFO:teuthology.orchestra.run.trial110.stdout:PLATFORM_ID="platform:el9"
2026-02-12T22:48:25.282 INFO:teuthology.orchestra.run.trial110.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T22:48:25.282 INFO:teuthology.orchestra.run.trial110.stdout:ANSI_COLOR="0;31"
2026-02-12T22:48:25.282 INFO:teuthology.orchestra.run.trial110.stdout:LOGO="fedora-logo-icon"
2026-02-12T22:48:25.282 INFO:teuthology.orchestra.run.trial110.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T22:48:25.282 INFO:teuthology.orchestra.run.trial110.stdout:HOME_URL="https://centos.org/"
2026-02-12T22:48:25.282 INFO:teuthology.orchestra.run.trial110.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T22:48:25.282 INFO:teuthology.orchestra.run.trial110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T22:48:25.282 INFO:teuthology.orchestra.run.trial110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T22:48:25.283 INFO:teuthology.provision.fog.trial110:Deploy complete!
2026-02-12T22:48:25.284 DEBUG:teuthology.parallel:result is None
2026-02-12T22:48:25.807 INFO:teuthology.lock.ops:Checking trial048.front.sepia.ceph.com
2026-02-12T22:48:25.807 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-12T22:48:25.917 INFO:teuthology.lock.ops:Checking trial110.front.sepia.ceph.com
2026-02-12T22:48:25.917 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-12T22:48:25.977 INFO:teuthology.lock.ops:Checking trial156.front.sepia.ceph.com
2026-02-12T22:48:25.977 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-12T22:48:26.046 INFO:teuthology.lock.ops:Updating [trial048.front.sepia.ceph.com]: set os type and version on server
2026-02-12T22:48:26.046 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial048.front.sepia.ceph.com'
2026-02-12T22:48:26.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial048.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T22:48:26.118 DEBUG:teuthology.orchestra.run.trial048:> true
2026-02-12T22:48:26.261 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial048.front.sepia.ceph.com'
2026-02-12T22:48:26.261 DEBUG:teuthology.orchestra.run.trial048:> uname -m
2026-02-12T22:48:26.317 INFO:teuthology.orchestra.run.trial048.stdout:x86_64
2026-02-12T22:48:26.318 DEBUG:teuthology.orchestra.run.trial048:> cat /etc/os-release
2026-02-12T22:48:26.374 INFO:teuthology.orchestra.run.trial048.stdout:NAME="CentOS Stream"
2026-02-12T22:48:26.374 INFO:teuthology.orchestra.run.trial048.stdout:VERSION="9"
2026-02-12T22:48:26.374 INFO:teuthology.orchestra.run.trial048.stdout:ID="centos"
2026-02-12T22:48:26.374 INFO:teuthology.orchestra.run.trial048.stdout:ID_LIKE="rhel fedora"
2026-02-12T22:48:26.374 INFO:teuthology.orchestra.run.trial048.stdout:VERSION_ID="9"
2026-02-12T22:48:26.375 INFO:teuthology.orchestra.run.trial048.stdout:PLATFORM_ID="platform:el9"
2026-02-12T22:48:26.375 INFO:teuthology.orchestra.run.trial048.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T22:48:26.375 INFO:teuthology.orchestra.run.trial048.stdout:ANSI_COLOR="0;31"
2026-02-12T22:48:26.375 INFO:teuthology.orchestra.run.trial048.stdout:LOGO="fedora-logo-icon"
2026-02-12T22:48:26.375 INFO:teuthology.orchestra.run.trial048.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T22:48:26.375 INFO:teuthology.orchestra.run.trial048.stdout:HOME_URL="https://centos.org/"
2026-02-12T22:48:26.375 INFO:teuthology.orchestra.run.trial048.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T22:48:26.375 INFO:teuthology.orchestra.run.trial048.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T22:48:26.375 INFO:teuthology.orchestra.run.trial048.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T22:48:26.376 INFO:teuthology.lock.ops:Updating trial048.front.sepia.ceph.com on lock server
2026-02-12T22:48:26.480 INFO:teuthology.lock.ops:Updating [trial110.front.sepia.ceph.com]: set os type and version on server
2026-02-12T22:48:26.480 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial110.front.sepia.ceph.com'
2026-02-12T22:48:26.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T22:48:26.547 DEBUG:teuthology.orchestra.run.trial110:> true
2026-02-12T22:48:26.621 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial110.front.sepia.ceph.com'
2026-02-12T22:48:26.621 DEBUG:teuthology.orchestra.run.trial110:> uname -m
2026-02-12T22:48:26.677 INFO:teuthology.orchestra.run.trial110.stdout:x86_64
2026-02-12T22:48:26.678 DEBUG:teuthology.orchestra.run.trial110:> cat /etc/os-release
2026-02-12T22:48:26.733 INFO:teuthology.orchestra.run.trial110.stdout:NAME="CentOS Stream"
2026-02-12T22:48:26.733 INFO:teuthology.orchestra.run.trial110.stdout:VERSION="9"
2026-02-12T22:48:26.733 INFO:teuthology.orchestra.run.trial110.stdout:ID="centos"
2026-02-12T22:48:26.733 INFO:teuthology.orchestra.run.trial110.stdout:ID_LIKE="rhel fedora"
2026-02-12T22:48:26.733 INFO:teuthology.orchestra.run.trial110.stdout:VERSION_ID="9"
2026-02-12T22:48:26.734 INFO:teuthology.orchestra.run.trial110.stdout:PLATFORM_ID="platform:el9"
2026-02-12T22:48:26.734 INFO:teuthology.orchestra.run.trial110.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T22:48:26.734 INFO:teuthology.orchestra.run.trial110.stdout:ANSI_COLOR="0;31"
2026-02-12T22:48:26.734 INFO:teuthology.orchestra.run.trial110.stdout:LOGO="fedora-logo-icon"
2026-02-12T22:48:26.734 INFO:teuthology.orchestra.run.trial110.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T22:48:26.734 INFO:teuthology.orchestra.run.trial110.stdout:HOME_URL="https://centos.org/"
2026-02-12T22:48:26.734 INFO:teuthology.orchestra.run.trial110.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T22:48:26.734 INFO:teuthology.orchestra.run.trial110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T22:48:26.734 INFO:teuthology.orchestra.run.trial110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T22:48:26.735 INFO:teuthology.lock.ops:Updating trial110.front.sepia.ceph.com on lock server
2026-02-12T22:48:26.838 INFO:teuthology.lock.ops:Updating [trial156.front.sepia.ceph.com]: set os type and version on server
2026-02-12T22:48:26.838 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial156.front.sepia.ceph.com'
2026-02-12T22:48:26.838 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T22:48:26.908 DEBUG:teuthology.orchestra.run.trial156:> true
2026-02-12T22:48:27.049 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial156.front.sepia.ceph.com'
2026-02-12T22:48:27.049 DEBUG:teuthology.orchestra.run.trial156:> uname -m
2026-02-12T22:48:27.105 INFO:teuthology.orchestra.run.trial156.stdout:x86_64
2026-02-12T22:48:27.106 DEBUG:teuthology.orchestra.run.trial156:> cat /etc/os-release
2026-02-12T22:48:27.162 INFO:teuthology.orchestra.run.trial156.stdout:NAME="CentOS Stream"
2026-02-12T22:48:27.163 INFO:teuthology.orchestra.run.trial156.stdout:VERSION="9"
2026-02-12T22:48:27.163 INFO:teuthology.orchestra.run.trial156.stdout:ID="centos"
2026-02-12T22:48:27.163 INFO:teuthology.orchestra.run.trial156.stdout:ID_LIKE="rhel fedora"
2026-02-12T22:48:27.163 INFO:teuthology.orchestra.run.trial156.stdout:VERSION_ID="9"
2026-02-12T22:48:27.163 INFO:teuthology.orchestra.run.trial156.stdout:PLATFORM_ID="platform:el9"
2026-02-12T22:48:27.163 INFO:teuthology.orchestra.run.trial156.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T22:48:27.164 INFO:teuthology.orchestra.run.trial156.stdout:ANSI_COLOR="0;31"
2026-02-12T22:48:27.164 INFO:teuthology.orchestra.run.trial156.stdout:LOGO="fedora-logo-icon"
2026-02-12T22:48:27.164 INFO:teuthology.orchestra.run.trial156.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T22:48:27.164 INFO:teuthology.orchestra.run.trial156.stdout:HOME_URL="https://centos.org/"
2026-02-12T22:48:27.164 INFO:teuthology.orchestra.run.trial156.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T22:48:27.164 INFO:teuthology.orchestra.run.trial156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T22:48:27.164 INFO:teuthology.orchestra.run.trial156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T22:48:27.165 INFO:teuthology.lock.ops:Updating trial156.front.sepia.ceph.com on lock server
2026-02-12T22:48:27.285 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T22:48:27.374 INFO:teuthology.dispatcher.supervisor:Running job 47308
2026-02-12T22:48:27.374 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_3e9168473cad21309cea53f654286d78057913dd/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/pdonnell-2026-02-12_20:30:49-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47308 --name pdonnell-2026-02-12_20:30:49-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial --description fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/squid 1-client 2-upgrade 3-compat_client/no}} -- /home/teuthworker/mnt/teuthology/pdonnell-2026-02-12_20:30:49-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47308/orig.config.yaml
2026-02-12T22:48:27.374 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/pdonnell-2026-02-12_20:30:49-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47308
2026-02-12T22:48:27.374 INFO:teuthology.dispatcher.supervisor:Job PID: 827001
2026-02-12T22:48:27.374 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-12T22:50:27.386 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T22:52:27.517 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T22:54:27.675 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T22:56:27.853 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T22:56:27.943 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2026-02-12T22:56:28.112 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-12T22:56:28.833 INFO:teuthology.orchestra.console.trial048:Power off
2026-02-12T22:56:28.833 DEBUG:teuthology.orchestra.console.trial048:pexpect command: ipmitool -H trial048.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T22:56:28.845 INFO:teuthology.orchestra.console.trial110:Power off
2026-02-12T22:56:28.845 DEBUG:teuthology.orchestra.console.trial110:pexpect command: ipmitool -H trial110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T22:56:28.855 INFO:teuthology.orchestra.console.trial156:Power off
2026-02-12T22:56:28.855 DEBUG:teuthology.orchestra.console.trial156:pexpect command: ipmitool -H trial156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T22:56:28.872 DEBUG:teuthology.orchestra.console.trial048:power off output: Chassis Power Control: Down/Off
2026-02-12T22:56:28.882 DEBUG:teuthology.orchestra.console.trial110:power off output: Chassis Power Control: Down/Off
2026-02-12T22:56:28.893 DEBUG:teuthology.orchestra.console.trial156:power off output: Chassis Power Control: Down/Off
2026-02-12T22:56:32.877 DEBUG:teuthology.orchestra.console.trial048:pexpect command: ipmitool -H trial048.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T22:56:32.883 DEBUG:teuthology.orchestra.console.trial110:pexpect command: ipmitool -H trial110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T22:56:32.894 DEBUG:teuthology.orchestra.console.trial156:pexpect command: ipmitool -H trial156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T22:56:32.985 DEBUG:teuthology.orchestra.console.trial048:check power output: Chassis Power is on
2026-02-12T22:56:32.991 DEBUG:teuthology.orchestra.console.trial110:check power output: Chassis Power is on
2026-02-12T22:56:33.002 DEBUG:teuthology.orchestra.console.trial156:check power output: Chassis Power is on
2026-02-12T22:56:36.990 DEBUG:teuthology.orchestra.console.trial048:pexpect command: ipmitool -H trial048.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T22:56:36.998 DEBUG:teuthology.orchestra.console.trial110:pexpect command: ipmitool -H trial110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T22:56:37.005 DEBUG:teuthology.orchestra.console.trial156:pexpect command: ipmitool -H trial156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T22:56:37.099 DEBUG:teuthology.orchestra.console.trial048:check power output: Chassis Power is off
2026-02-12T22:56:37.100 INFO:teuthology.orchestra.console.trial048:Power off completed
2026-02-12T22:56:37.106 DEBUG:teuthology.orchestra.console.trial110:check power output: Chassis Power is off
2026-02-12T22:56:37.107 INFO:teuthology.orchestra.console.trial110:Power off completed
2026-02-12T22:56:37.114 DEBUG:teuthology.orchestra.console.trial156:check power output: Chassis Power is off
2026-02-12T22:56:37.114 INFO:teuthology.orchestra.console.trial156:Power off completed
2026-02-12T22:56:37.392 INFO:teuthology.lock.ops:unlocked: trial048.front.sepia.ceph.com
2026-02-12T22:56:37.400 INFO:teuthology.lock.ops:unlocked: trial110.front.sepia.ceph.com
2026-02-12T22:56:37.411 INFO:teuthology.lock.ops:unlocked: trial156.front.sepia.ceph.com