2026-02-13T02:12:00.254 INFO:root:teuthology version: 1.2.4.dev9+ga2c505dae
2026-02-13T02:12:00.254 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T02:12:00.326 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T02:12:00.579 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7816ec93b290>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7816ec12c5e0>, signals=[15])
2026-02-13T02:12:00.579 INFO:teuthology.lock.ops:Start node 'trial077.front.sepia.ceph.com' reimaging
2026-02-13T02:12:00.579 INFO:teuthology.lock.ops:Updating [trial077.front.sepia.ceph.com]: reset os type and version on server
2026-02-13T02:12:00.579 INFO:teuthology.lock.ops:Updating trial077.front.sepia.ceph.com on lock server
2026-02-13T02:12:00.636 INFO:teuthology.lock.ops:Start node 'trial093.front.sepia.ceph.com' reimaging
2026-02-13T02:12:00.636 INFO:teuthology.lock.ops:Updating [trial093.front.sepia.ceph.com]: reset os type and version on server
2026-02-13T02:12:00.636 INFO:teuthology.lock.ops:Updating trial093.front.sepia.ceph.com on lock server
2026-02-13T02:12:00.694 INFO:teuthology.lock.ops:Start node 'trial115.front.sepia.ceph.com' reimaging
2026-02-13T02:12:00.694 INFO:teuthology.lock.ops:Updating [trial115.front.sepia.ceph.com]: reset os type and version on server
2026-02-13T02:12:00.694 INFO:teuthology.lock.ops:Updating trial115.front.sepia.ceph.com on lock server
2026-02-13T02:12:00.937 DEBUG:teuthology.provision.fog.trial093:Requesting image trial_rocky_10.1 (ID 4)
2026-02-13T02:12:00.937 DEBUG:teuthology.provision.fog.trial077:Requesting image trial_rocky_10.1 (ID 4)
2026-02-13T02:12:00.990 INFO:teuthology.provision.fog.trial093:Scheduling deploy of rocky 10.1
2026-02-13T02:12:00.992 INFO:teuthology.provision.fog.trial077:Scheduling deploy of rocky 10.1
2026-02-13T02:12:00.995 DEBUG:teuthology.provision.fog.trial115:Requesting image trial_rocky_10.1 (ID 4)
2026-02-13T02:12:01.056 INFO:teuthology.provision.fog.trial115:Scheduling deploy of rocky 10.1
2026-02-13T02:12:02.302 INFO:teuthology.orchestra.console.trial093:Power off
2026-02-13T02:12:02.302 DEBUG:teuthology.orchestra.console.trial093:pexpect command: ipmitool -H trial093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-13T02:12:02.313 INFO:teuthology.orchestra.console.trial077:Power off
2026-02-13T02:12:02.314 DEBUG:teuthology.orchestra.console.trial077:pexpect command: ipmitool -H trial077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-13T02:12:02.323 INFO:teuthology.orchestra.console.trial115:Power off
2026-02-13T02:12:02.323 DEBUG:teuthology.orchestra.console.trial115:pexpect command: ipmitool -H trial115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-13T02:12:02.340 DEBUG:teuthology.orchestra.console.trial093:power off output: Chassis Power Control: Down/Off
2026-02-13T02:12:02.351 DEBUG:teuthology.orchestra.console.trial077:power off output: Chassis Power Control: Down/Off
2026-02-13T02:12:02.361 DEBUG:teuthology.orchestra.console.trial115:power off output: Chassis Power Control: Down/Off
2026-02-13T02:12:06.345 DEBUG:teuthology.orchestra.console.trial093:pexpect command: ipmitool -H trial093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T02:12:06.351 DEBUG:teuthology.orchestra.console.trial077:pexpect command: ipmitool -H trial077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T02:12:06.363 DEBUG:teuthology.orchestra.console.trial115:pexpect command: ipmitool -H trial115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T02:12:06.453 DEBUG:teuthology.orchestra.console.trial093:check power output: Chassis Power is off
2026-02-13T02:12:06.454 INFO:teuthology.orchestra.console.trial093:Power off completed
2026-02-13T02:12:06.459 DEBUG:teuthology.orchestra.console.trial077:check power output: Chassis Power is off
2026-02-13T02:12:06.459 INFO:teuthology.orchestra.console.trial077:Power off completed
2026-02-13T02:12:06.471 DEBUG:teuthology.orchestra.console.trial115:check power output: Chassis Power is off
2026-02-13T02:12:06.471 INFO:teuthology.orchestra.console.trial115:Power off completed
2026-02-13T02:12:06.554 INFO:teuthology.orchestra.console.trial093:Power on
2026-02-13T02:12:06.555 DEBUG:teuthology.orchestra.console.trial093:pexpect command: ipmitool -H trial093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-13T02:12:06.563 INFO:teuthology.orchestra.console.trial077:Power on
2026-02-13T02:12:06.564 DEBUG:teuthology.orchestra.console.trial077:pexpect command: ipmitool -H trial077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-13T02:12:06.572 INFO:teuthology.orchestra.console.trial115:Power on
2026-02-13T02:12:06.572 DEBUG:teuthology.orchestra.console.trial115:pexpect command: ipmitool -H trial115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-13T02:12:06.594 DEBUG:teuthology.orchestra.console.trial093:power on output: Chassis Power Control: Up/On
2026-02-13T02:12:06.606 DEBUG:teuthology.orchestra.console.trial077:power on output: Chassis Power Control: Up/On
2026-02-13T02:12:06.611 DEBUG:teuthology.orchestra.console.trial115:power on output: Chassis Power Control: Up/On
2026-02-13T02:12:10.599 DEBUG:teuthology.orchestra.console.trial093:pexpect command: ipmitool -H trial093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T02:12:10.608 DEBUG:teuthology.orchestra.console.trial077:pexpect command: ipmitool -H trial077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T02:12:10.616 DEBUG:teuthology.orchestra.console.trial115:pexpect command: ipmitool -H trial115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T02:12:10.711 DEBUG:teuthology.orchestra.console.trial093:check power output: Chassis Power is on
2026-02-13T02:12:10.711 INFO:teuthology.orchestra.console.trial093:Power on completed
2026-02-13T02:12:10.717 DEBUG:teuthology.orchestra.console.trial077:check power output: Chassis Power is on
2026-02-13T02:12:10.717 INFO:teuthology.orchestra.console.trial077:Power on completed
2026-02-13T02:12:10.725 DEBUG:teuthology.orchestra.console.trial115:check power output: Chassis Power is off
2026-02-13T02:12:10.812 INFO:teuthology.provision.fog.trial093:Waiting for deploy to finish
2026-02-13T02:12:10.818 INFO:teuthology.provision.fog.trial077:Waiting for deploy to finish
2026-02-13T02:12:14.728 DEBUG:teuthology.orchestra.console.trial115:pexpect command: ipmitool -H trial115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T02:12:14.837 DEBUG:teuthology.orchestra.console.trial115:check power output: Chassis Power is on
2026-02-13T02:12:14.838 INFO:teuthology.orchestra.console.trial115:Power on completed
2026-02-13T02:12:14.938 INFO:teuthology.provision.fog.trial115:Waiting for deploy to finish
2026-02-13T02:13:58.269 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T02:13:58.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T02:14:02.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T02:14:05.085 WARNING:teuthology.provision.fog.trial115:[Errno None] Unable to connect to port 22 on 10.20.193.115
2026-02-13T02:14:11.092 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T02:14:12.573 WARNING:teuthology.provision.fog.trial093:[Errno None] Unable to connect to port 22 on 10.20.193.93
2026-02-13T02:14:12.573 WARNING:teuthology.provision.fog.trial077:[Errno None] Unable to connect to port 22 on 10.20.193.77
2026-02-13T02:14:14.173 WARNING:teuthology.provision.fog.trial115:[Errno None] Unable to connect to port 22 on 10.20.193.115
2026-02-13T02:14:18.579 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T02:14:18.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T02:14:20.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T02:14:21.661 WARNING:teuthology.provision.fog.trial093:[Errno None] Unable to connect to port 22 on 10.20.193.93
2026-02-13T02:14:21.661 WARNING:teuthology.provision.fog.trial077:[Errno None] Unable to connect to port 22 on 10.20.193.77
2026-02-13T02:14:23.261 WARNING:teuthology.provision.fog.trial115:[Errno None] Unable to connect to port 22 on 10.20.193.115
2026-02-13T02:14:27.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T02:14:27.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T02:14:29.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T02:14:30.749 WARNING:teuthology.provision.fog.trial093:[Errno None] Unable to connect to port 22 on 10.20.193.93
2026-02-13T02:14:30.749 WARNING:teuthology.provision.fog.trial077:[Errno None] Unable to connect to port 22 on 10.20.193.77
2026-02-13T02:14:32.349 WARNING:teuthology.provision.fog.trial115:[Errno None] Unable to connect to port 22 on 10.20.193.115
2026-02-13T02:14:36.754 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T02:14:36.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T02:14:38.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T02:15:12.960 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-13T02:15:12.960 DEBUG:teuthology.orchestra.run.trial077:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-13T02:15:12.963 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-13T02:15:12.963 DEBUG:teuthology.orchestra.run.trial093:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-13T02:15:14.463 ERROR:teuthology.orchestra.connection:Error authenticating with trial115.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-13T02:15:18.148 INFO:teuthology.provision.fog.trial077:Node is ready
2026-02-13T02:15:18.148 DEBUG:teuthology.orchestra.run.trial077:> hostname
2026-02-13T02:15:18.156 INFO:teuthology.provision.fog.trial093:Node is ready
2026-02-13T02:15:18.157 DEBUG:teuthology.orchestra.run.trial093:> hostname
2026-02-13T02:15:18.166 INFO:teuthology.orchestra.run.trial077.stdout:trial077.front.sepia.ceph.com
2026-02-13T02:15:18.167 DEBUG:teuthology.orchestra.run.trial077:> grep trial077.front.sepia.ceph.com /etc/hosts
2026-02-13T02:15:18.175 INFO:teuthology.orchestra.run.trial093.stdout:trial093.front.sepia.ceph.com
2026-02-13T02:15:18.175 DEBUG:teuthology.orchestra.run.trial093:> grep trial093.front.sepia.ceph.com /etc/hosts
2026-02-13T02:15:18.224 INFO:teuthology.orchestra.run.trial077.stdout:10.20.193.77 trial077.front.sepia.ceph.com trial077
2026-02-13T02:15:18.225 DEBUG:teuthology.orchestra.run.trial077:> sudo hostname trial077
2026-02-13T02:15:18.233 INFO:teuthology.orchestra.run.trial093.stdout:10.20.193.93 trial093.front.sepia.ceph.com trial093
2026-02-13T02:15:18.233 DEBUG:teuthology.orchestra.run.trial093:> sudo hostname trial093
2026-02-13T02:15:18.299 DEBUG:teuthology.orchestra.run.trial077:> sudo sed -i -e 's/trial077.front.sepia.ceph.com/trial077/g' /etc/hosts
2026-02-13T02:15:18.306 DEBUG:teuthology.orchestra.run.trial093:> sudo sed -i -e 's/trial093.front.sepia.ceph.com/trial093/g' /etc/hosts
2026-02-13T02:15:18.366 DEBUG:teuthology.orchestra.run.trial077:> sudo sed -i -e 's/10.20.193.77/10.20.193.77/g' /etc/hosts
2026-02-13T02:15:18.373 DEBUG:teuthology.orchestra.run.trial093:> sudo sed -i -e 's/10.20.193.93/10.20.193.93/g' /etc/hosts
2026-02-13T02:15:18.433 DEBUG:teuthology.orchestra.run.trial077:> sudo sed -i -e 's/trial077.front.sepia.ceph.com/trial077/g' /etc/hostname
2026-02-13T02:15:18.439 DEBUG:teuthology.orchestra.run.trial093:> sudo sed -i -e 's/trial093.front.sepia.ceph.com/trial093/g' /etc/hostname
2026-02-13T02:15:18.563 DEBUG:teuthology.orchestra.run.trial077:> sudo hostname trial077
2026-02-13T02:15:18.573 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-13T02:15:18.573 DEBUG:teuthology.orchestra.run.trial115:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-13T02:15:18.588 DEBUG:teuthology.orchestra.run.trial093:> sudo hostname trial093
2026-02-13T02:15:18.628 DEBUG:teuthology.orchestra.run.trial077:> cat /etc/os-release
2026-02-13T02:15:18.654 DEBUG:teuthology.orchestra.run.trial093:> cat /etc/os-release
2026-02-13T02:15:18.687 INFO:teuthology.orchestra.run.trial077.stdout:NAME="Rocky Linux"
2026-02-13T02:15:18.687 INFO:teuthology.orchestra.run.trial077.stdout:VERSION="10.1 (Red Quartz)"
2026-02-13T02:15:18.687 INFO:teuthology.orchestra.run.trial077.stdout:ID="rocky"
2026-02-13T02:15:18.687 INFO:teuthology.orchestra.run.trial077.stdout:ID_LIKE="rhel centos fedora"
2026-02-13T02:15:18.687 INFO:teuthology.orchestra.run.trial077.stdout:VERSION_ID="10.1"
2026-02-13T02:15:18.687 INFO:teuthology.orchestra.run.trial077.stdout:PLATFORM_ID="platform:el10"
2026-02-13T02:15:18.687 INFO:teuthology.orchestra.run.trial077.stdout:PRETTY_NAME="Rocky Linux 10.1 (Red Quartz)"
2026-02-13T02:15:18.687 INFO:teuthology.orchestra.run.trial077.stdout:ANSI_COLOR="0;32"
2026-02-13T02:15:18.688 INFO:teuthology.orchestra.run.trial077.stdout:LOGO="fedora-logo-icon"
2026-02-13T02:15:18.688 INFO:teuthology.orchestra.run.trial077.stdout:CPE_NAME="cpe:/o:rocky:rocky:10::baseos"
2026-02-13T02:15:18.688 INFO:teuthology.orchestra.run.trial077.stdout:HOME_URL="https://rockylinux.org/"
2026-02-13T02:15:18.688 INFO:teuthology.orchestra.run.trial077.stdout:VENDOR_NAME="RESF"
2026-02-13T02:15:18.688 INFO:teuthology.orchestra.run.trial077.stdout:VENDOR_URL="https://resf.org/"
2026-02-13T02:15:18.688 INFO:teuthology.orchestra.run.trial077.stdout:BUG_REPORT_URL="https://bugs.rockylinux.org/"
2026-02-13T02:15:18.688 INFO:teuthology.orchestra.run.trial077.stdout:SUPPORT_END="2035-05-31"
2026-02-13T02:15:18.688 INFO:teuthology.orchestra.run.trial077.stdout:ROCKY_SUPPORT_PRODUCT="Rocky-Linux-10"
2026-02-13T02:15:18.688 INFO:teuthology.orchestra.run.trial077.stdout:ROCKY_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T02:15:18.688 INFO:teuthology.orchestra.run.trial077.stdout:REDHAT_SUPPORT_PRODUCT="Rocky Linux"
2026-02-13T02:15:18.688 INFO:teuthology.orchestra.run.trial077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T02:15:18.689 INFO:teuthology.provision.fog.trial077:Deploy complete!
2026-02-13T02:15:18.690 DEBUG:teuthology.parallel:result is None
2026-02-13T02:15:18.712 INFO:teuthology.orchestra.run.trial093.stdout:NAME="Rocky Linux"
2026-02-13T02:15:18.712 INFO:teuthology.orchestra.run.trial093.stdout:VERSION="10.1 (Red Quartz)"
2026-02-13T02:15:18.712 INFO:teuthology.orchestra.run.trial093.stdout:ID="rocky"
2026-02-13T02:15:18.712 INFO:teuthology.orchestra.run.trial093.stdout:ID_LIKE="rhel centos fedora"
2026-02-13T02:15:18.713 INFO:teuthology.orchestra.run.trial093.stdout:VERSION_ID="10.1"
2026-02-13T02:15:18.713 INFO:teuthology.orchestra.run.trial093.stdout:PLATFORM_ID="platform:el10"
2026-02-13T02:15:18.713 INFO:teuthology.orchestra.run.trial093.stdout:PRETTY_NAME="Rocky Linux 10.1 (Red Quartz)"
2026-02-13T02:15:18.713 INFO:teuthology.orchestra.run.trial093.stdout:ANSI_COLOR="0;32"
2026-02-13T02:15:18.713 INFO:teuthology.orchestra.run.trial093.stdout:LOGO="fedora-logo-icon"
2026-02-13T02:15:18.713 INFO:teuthology.orchestra.run.trial093.stdout:CPE_NAME="cpe:/o:rocky:rocky:10::baseos"
2026-02-13T02:15:18.713 INFO:teuthology.orchestra.run.trial093.stdout:HOME_URL="https://rockylinux.org/"
2026-02-13T02:15:18.713 INFO:teuthology.orchestra.run.trial093.stdout:VENDOR_NAME="RESF"
2026-02-13T02:15:18.713 INFO:teuthology.orchestra.run.trial093.stdout:VENDOR_URL="https://resf.org/"
2026-02-13T02:15:18.713 INFO:teuthology.orchestra.run.trial093.stdout:BUG_REPORT_URL="https://bugs.rockylinux.org/"
2026-02-13T02:15:18.713 INFO:teuthology.orchestra.run.trial093.stdout:SUPPORT_END="2035-05-31"
2026-02-13T02:15:18.714 INFO:teuthology.orchestra.run.trial093.stdout:ROCKY_SUPPORT_PRODUCT="Rocky-Linux-10"
2026-02-13T02:15:18.714 INFO:teuthology.orchestra.run.trial093.stdout:ROCKY_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T02:15:18.714 INFO:teuthology.orchestra.run.trial093.stdout:REDHAT_SUPPORT_PRODUCT="Rocky Linux"
2026-02-13T02:15:18.714 INFO:teuthology.orchestra.run.trial093.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T02:15:18.714 INFO:teuthology.provision.fog.trial093:Deploy complete!
2026-02-13T02:15:18.715 DEBUG:teuthology.parallel:result is None
2026-02-13T02:15:23.777 INFO:teuthology.provision.fog.trial115:Node is ready
2026-02-13T02:15:23.777 DEBUG:teuthology.orchestra.run.trial115:> hostname
2026-02-13T02:15:23.834 INFO:teuthology.orchestra.run.trial115.stdout:trial115.front.sepia.ceph.com
2026-02-13T02:15:23.835 DEBUG:teuthology.orchestra.run.trial115:> grep trial115.front.sepia.ceph.com /etc/hosts
2026-02-13T02:15:23.892 INFO:teuthology.orchestra.run.trial115.stdout:10.20.193.115 trial115.front.sepia.ceph.com trial115
2026-02-13T02:15:23.893 DEBUG:teuthology.orchestra.run.trial115:> sudo hostname trial115
2026-02-13T02:15:23.966 DEBUG:teuthology.orchestra.run.trial115:> sudo sed -i -e 's/trial115.front.sepia.ceph.com/trial115/g' /etc/hosts
2026-02-13T02:15:24.033 DEBUG:teuthology.orchestra.run.trial115:> sudo sed -i -e 's/10.20.193.115/10.20.193.115/g' /etc/hosts
2026-02-13T02:15:24.101 DEBUG:teuthology.orchestra.run.trial115:> sudo sed -i -e 's/trial115.front.sepia.ceph.com/trial115/g' /etc/hostname
2026-02-13T02:15:24.168 DEBUG:teuthology.orchestra.run.trial115:> sudo hostname trial115
2026-02-13T02:15:24.234 DEBUG:teuthology.orchestra.run.trial115:> cat /etc/os-release
2026-02-13T02:15:24.291 INFO:teuthology.orchestra.run.trial115.stdout:NAME="Rocky Linux"
2026-02-13T02:15:24.291 INFO:teuthology.orchestra.run.trial115.stdout:VERSION="10.1 (Red Quartz)"
2026-02-13T02:15:24.291 INFO:teuthology.orchestra.run.trial115.stdout:ID="rocky"
2026-02-13T02:15:24.291 INFO:teuthology.orchestra.run.trial115.stdout:ID_LIKE="rhel centos fedora"
2026-02-13T02:15:24.291 INFO:teuthology.orchestra.run.trial115.stdout:VERSION_ID="10.1"
2026-02-13T02:15:24.292 INFO:teuthology.orchestra.run.trial115.stdout:PLATFORM_ID="platform:el10"
2026-02-13T02:15:24.292 INFO:teuthology.orchestra.run.trial115.stdout:PRETTY_NAME="Rocky Linux 10.1 (Red Quartz)"
2026-02-13T02:15:24.292 INFO:teuthology.orchestra.run.trial115.stdout:ANSI_COLOR="0;32"
2026-02-13T02:15:24.292 INFO:teuthology.orchestra.run.trial115.stdout:LOGO="fedora-logo-icon"
2026-02-13T02:15:24.292 INFO:teuthology.orchestra.run.trial115.stdout:CPE_NAME="cpe:/o:rocky:rocky:10::baseos"
2026-02-13T02:15:24.292 INFO:teuthology.orchestra.run.trial115.stdout:HOME_URL="https://rockylinux.org/"
2026-02-13T02:15:24.292 INFO:teuthology.orchestra.run.trial115.stdout:VENDOR_NAME="RESF"
2026-02-13T02:15:24.292 INFO:teuthology.orchestra.run.trial115.stdout:VENDOR_URL="https://resf.org/"
2026-02-13T02:15:24.292 INFO:teuthology.orchestra.run.trial115.stdout:BUG_REPORT_URL="https://bugs.rockylinux.org/"
2026-02-13T02:15:24.292 INFO:teuthology.orchestra.run.trial115.stdout:SUPPORT_END="2035-05-31"
2026-02-13T02:15:24.292 INFO:teuthology.orchestra.run.trial115.stdout:ROCKY_SUPPORT_PRODUCT="Rocky-Linux-10"
2026-02-13T02:15:24.292 INFO:teuthology.orchestra.run.trial115.stdout:ROCKY_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T02:15:24.293 INFO:teuthology.orchestra.run.trial115.stdout:REDHAT_SUPPORT_PRODUCT="Rocky Linux"
2026-02-13T02:15:24.293 INFO:teuthology.orchestra.run.trial115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T02:15:24.293 INFO:teuthology.provision.fog.trial115:Deploy complete!
2026-02-13T02:15:24.294 DEBUG:teuthology.parallel:result is None
2026-02-13T02:15:24.820 INFO:teuthology.lock.ops:Checking trial077.front.sepia.ceph.com
2026-02-13T02:15:24.821 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-13T02:15:24.928 INFO:teuthology.lock.ops:Checking trial093.front.sepia.ceph.com
2026-02-13T02:15:24.928 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-13T02:15:24.990 INFO:teuthology.lock.ops:Checking trial115.front.sepia.ceph.com
2026-02-13T02:15:24.990 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-13T02:15:25.046 INFO:teuthology.lock.ops:Updating [trial077.front.sepia.ceph.com]: set os type and version on server
2026-02-13T02:15:25.046 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial077.front.sepia.ceph.com'
2026-02-13T02:15:25.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T02:15:25.106 DEBUG:teuthology.orchestra.run.trial077:> true
2026-02-13T02:15:25.193 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial077.front.sepia.ceph.com'
2026-02-13T02:15:25.193 DEBUG:teuthology.orchestra.run.trial077:> uname -m
2026-02-13T02:15:25.250 INFO:teuthology.orchestra.run.trial077.stdout:x86_64
2026-02-13T02:15:25.250 DEBUG:teuthology.orchestra.run.trial077:> cat /etc/os-release
2026-02-13T02:15:25.308 INFO:teuthology.orchestra.run.trial077.stdout:NAME="Rocky Linux"
2026-02-13T02:15:25.308 INFO:teuthology.orchestra.run.trial077.stdout:VERSION="10.1 (Red Quartz)"
2026-02-13T02:15:25.308 INFO:teuthology.orchestra.run.trial077.stdout:ID="rocky"
2026-02-13T02:15:25.308 INFO:teuthology.orchestra.run.trial077.stdout:ID_LIKE="rhel centos fedora"
2026-02-13T02:15:25.308 INFO:teuthology.orchestra.run.trial077.stdout:VERSION_ID="10.1"
2026-02-13T02:15:25.308 INFO:teuthology.orchestra.run.trial077.stdout:PLATFORM_ID="platform:el10"
2026-02-13T02:15:25.309 INFO:teuthology.orchestra.run.trial077.stdout:PRETTY_NAME="Rocky Linux 10.1 (Red Quartz)"
2026-02-13T02:15:25.309 INFO:teuthology.orchestra.run.trial077.stdout:ANSI_COLOR="0;32"
2026-02-13T02:15:25.309 INFO:teuthology.orchestra.run.trial077.stdout:LOGO="fedora-logo-icon"
2026-02-13T02:15:25.309 INFO:teuthology.orchestra.run.trial077.stdout:CPE_NAME="cpe:/o:rocky:rocky:10::baseos"
2026-02-13T02:15:25.309 INFO:teuthology.orchestra.run.trial077.stdout:HOME_URL="https://rockylinux.org/"
2026-02-13T02:15:25.309 INFO:teuthology.orchestra.run.trial077.stdout:VENDOR_NAME="RESF"
2026-02-13T02:15:25.309 INFO:teuthology.orchestra.run.trial077.stdout:VENDOR_URL="https://resf.org/"
2026-02-13T02:15:25.309 INFO:teuthology.orchestra.run.trial077.stdout:BUG_REPORT_URL="https://bugs.rockylinux.org/"
2026-02-13T02:15:25.309 INFO:teuthology.orchestra.run.trial077.stdout:SUPPORT_END="2035-05-31"
2026-02-13T02:15:25.309 INFO:teuthology.orchestra.run.trial077.stdout:ROCKY_SUPPORT_PRODUCT="Rocky-Linux-10"
2026-02-13T02:15:25.310 INFO:teuthology.orchestra.run.trial077.stdout:ROCKY_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T02:15:25.310 INFO:teuthology.orchestra.run.trial077.stdout:REDHAT_SUPPORT_PRODUCT="Rocky Linux"
2026-02-13T02:15:25.310 INFO:teuthology.orchestra.run.trial077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T02:15:25.310 INFO:teuthology.lock.ops:Updating trial077.front.sepia.ceph.com on lock server
2026-02-13T02:15:25.412 INFO:teuthology.lock.ops:Updating [trial093.front.sepia.ceph.com]: set os type and version on server
2026-02-13T02:15:25.412 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial093.front.sepia.ceph.com'
2026-02-13T02:15:25.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T02:15:25.474 DEBUG:teuthology.orchestra.run.trial093:> true
2026-02-13T02:15:25.552 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial093.front.sepia.ceph.com'
2026-02-13T02:15:25.553 DEBUG:teuthology.orchestra.run.trial093:> uname -m
2026-02-13T02:15:25.609 INFO:teuthology.orchestra.run.trial093.stdout:x86_64
2026-02-13T02:15:25.610 DEBUG:teuthology.orchestra.run.trial093:> cat /etc/os-release
2026-02-13T02:15:25.666 INFO:teuthology.orchestra.run.trial093.stdout:NAME="Rocky Linux"
2026-02-13T02:15:25.666 INFO:teuthology.orchestra.run.trial093.stdout:VERSION="10.1 (Red Quartz)"
2026-02-13T02:15:25.666 INFO:teuthology.orchestra.run.trial093.stdout:ID="rocky"
2026-02-13T02:15:25.666 INFO:teuthology.orchestra.run.trial093.stdout:ID_LIKE="rhel centos fedora"
2026-02-13T02:15:25.666 INFO:teuthology.orchestra.run.trial093.stdout:VERSION_ID="10.1"
2026-02-13T02:15:25.666 INFO:teuthology.orchestra.run.trial093.stdout:PLATFORM_ID="platform:el10"
2026-02-13T02:15:25.667 INFO:teuthology.orchestra.run.trial093.stdout:PRETTY_NAME="Rocky Linux 10.1 (Red Quartz)"
2026-02-13T02:15:25.667 INFO:teuthology.orchestra.run.trial093.stdout:ANSI_COLOR="0;32"
2026-02-13T02:15:25.667 INFO:teuthology.orchestra.run.trial093.stdout:LOGO="fedora-logo-icon"
2026-02-13T02:15:25.667 INFO:teuthology.orchestra.run.trial093.stdout:CPE_NAME="cpe:/o:rocky:rocky:10::baseos"
2026-02-13T02:15:25.667 INFO:teuthology.orchestra.run.trial093.stdout:HOME_URL="https://rockylinux.org/"
2026-02-13T02:15:25.667 INFO:teuthology.orchestra.run.trial093.stdout:VENDOR_NAME="RESF"
2026-02-13T02:15:25.667 INFO:teuthology.orchestra.run.trial093.stdout:VENDOR_URL="https://resf.org/"
2026-02-13T02:15:25.667 INFO:teuthology.orchestra.run.trial093.stdout:BUG_REPORT_URL="https://bugs.rockylinux.org/"
2026-02-13T02:15:25.667 INFO:teuthology.orchestra.run.trial093.stdout:SUPPORT_END="2035-05-31"
2026-02-13T02:15:25.667 INFO:teuthology.orchestra.run.trial093.stdout:ROCKY_SUPPORT_PRODUCT="Rocky-Linux-10"
2026-02-13T02:15:25.667 INFO:teuthology.orchestra.run.trial093.stdout:ROCKY_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T02:15:25.667 INFO:teuthology.orchestra.run.trial093.stdout:REDHAT_SUPPORT_PRODUCT="Rocky Linux"
2026-02-13T02:15:25.667 INFO:teuthology.orchestra.run.trial093.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T02:15:25.668 INFO:teuthology.lock.ops:Updating trial093.front.sepia.ceph.com on lock server
2026-02-13T02:15:25.763 INFO:teuthology.lock.ops:Updating [trial115.front.sepia.ceph.com]: set os type and version on server
2026-02-13T02:15:25.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial115.front.sepia.ceph.com'
2026-02-13T02:15:25.764 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T02:15:25.823 DEBUG:teuthology.orchestra.run.trial115:> true
2026-02-13T02:15:25.895 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial115.front.sepia.ceph.com'
2026-02-13T02:15:25.895 DEBUG:teuthology.orchestra.run.trial115:> uname -m
2026-02-13T02:15:25.953 INFO:teuthology.orchestra.run.trial115.stdout:x86_64
2026-02-13T02:15:25.953 DEBUG:teuthology.orchestra.run.trial115:> cat /etc/os-release
2026-02-13T02:15:26.009 INFO:teuthology.orchestra.run.trial115.stdout:NAME="Rocky Linux"
2026-02-13T02:15:26.009 INFO:teuthology.orchestra.run.trial115.stdout:VERSION="10.1 (Red Quartz)"
2026-02-13T02:15:26.009 INFO:teuthology.orchestra.run.trial115.stdout:ID="rocky"
2026-02-13T02:15:26.009 INFO:teuthology.orchestra.run.trial115.stdout:ID_LIKE="rhel centos fedora"
2026-02-13T02:15:26.010 INFO:teuthology.orchestra.run.trial115.stdout:VERSION_ID="10.1"
2026-02-13T02:15:26.010 INFO:teuthology.orchestra.run.trial115.stdout:PLATFORM_ID="platform:el10"
2026-02-13T02:15:26.010 INFO:teuthology.orchestra.run.trial115.stdout:PRETTY_NAME="Rocky Linux 10.1 (Red Quartz)"
2026-02-13T02:15:26.010 INFO:teuthology.orchestra.run.trial115.stdout:ANSI_COLOR="0;32"
2026-02-13T02:15:26.010 INFO:teuthology.orchestra.run.trial115.stdout:LOGO="fedora-logo-icon"
2026-02-13T02:15:26.010 INFO:teuthology.orchestra.run.trial115.stdout:CPE_NAME="cpe:/o:rocky:rocky:10::baseos"
2026-02-13T02:15:26.010 INFO:teuthology.orchestra.run.trial115.stdout:HOME_URL="https://rockylinux.org/"
2026-02-13T02:15:26.010 INFO:teuthology.orchestra.run.trial115.stdout:VENDOR_NAME="RESF"
2026-02-13T02:15:26.010 INFO:teuthology.orchestra.run.trial115.stdout:VENDOR_URL="https://resf.org/"
2026-02-13T02:15:26.010 INFO:teuthology.orchestra.run.trial115.stdout:BUG_REPORT_URL="https://bugs.rockylinux.org/"
2026-02-13T02:15:26.010 INFO:teuthology.orchestra.run.trial115.stdout:SUPPORT_END="2035-05-31"
2026-02-13T02:15:26.010 INFO:teuthology.orchestra.run.trial115.stdout:ROCKY_SUPPORT_PRODUCT="Rocky-Linux-10"
2026-02-13T02:15:26.010 INFO:teuthology.orchestra.run.trial115.stdout:ROCKY_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T02:15:26.010 INFO:teuthology.orchestra.run.trial115.stdout:REDHAT_SUPPORT_PRODUCT="Rocky Linux"
2026-02-13T02:15:26.010 INFO:teuthology.orchestra.run.trial115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T02:15:26.011 INFO:teuthology.lock.ops:Updating trial115.front.sepia.ceph.com on lock server
2026-02-13T02:15:26.097 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T02:15:26.197 INFO:teuthology.dispatcher.supervisor:Running job 47977
2026-02-13T02:15:26.197 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47977 --name pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial --description fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/fs/norstats}} -- /home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47977/orig.config.yaml
2026-02-13T02:15:26.198 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47977
2026-02-13T02:15:26.198 INFO:teuthology.dispatcher.supervisor:Job PID: 1252817
2026-02-13T02:15:26.198 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-13T02:17:26.282 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T02:19:26.405 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T02:21:26.592 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T02:23:26.776 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T02:25:26.954 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T02:27:27.146 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T02:29:27.301 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T02:29:27.380 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2026-02-13T02:29:27.550 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-13T02:29:28.216 INFO:teuthology.orchestra.console.trial093:Power off
2026-02-13T02:29:28.216 DEBUG:teuthology.orchestra.console.trial093:pexpect command: ipmitool -H trial093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-13T02:29:28.274 INFO:teuthology.orchestra.console.trial115:Power off
2026-02-13T02:29:28.274 DEBUG:teuthology.orchestra.console.trial115:pexpect command: ipmitool -H trial115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-13T02:29:28.279 DEBUG:teuthology.orchestra.console.trial093:power off output: Chassis Power Control: Down/Off
2026-02-13T02:29:28.292 INFO:teuthology.orchestra.console.trial077:Power off
2026-02-13T02:29:28.292 DEBUG:teuthology.orchestra.console.trial077:pexpect command: ipmitool -H trial077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-13T02:29:28.311 DEBUG:teuthology.orchestra.console.trial115:power off output: Chassis Power Control: Down/Off
2026-02-13T02:29:28.329 DEBUG:teuthology.orchestra.console.trial077:power off output: Chassis Power Control: Down/Off
2026-02-13T02:29:32.284 DEBUG:teuthology.orchestra.console.trial093:pexpect command: ipmitool -H trial093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T02:29:32.312 DEBUG:teuthology.orchestra.console.trial115:pexpect command: ipmitool -H trial115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T02:29:32.330 DEBUG:teuthology.orchestra.console.trial077:pexpect command: ipmitool -H trial077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T02:29:32.392 DEBUG:teuthology.orchestra.console.trial093:check power output: Chassis Power is on
2026-02-13T02:29:32.420 DEBUG:teuthology.orchestra.console.trial115:check power output: Chassis Power is on
2026-02-13T02:29:32.438 DEBUG:teuthology.orchestra.console.trial077:check power output: Chassis Power is on
2026-02-13T02:29:36.397 DEBUG:teuthology.orchestra.console.trial093:pexpect command: ipmitool -H trial093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T02:29:36.421 DEBUG:teuthology.orchestra.console.trial115:pexpect command: ipmitool -H trial115.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T02:29:36.439 DEBUG:teuthology.orchestra.console.trial077:pexpect command: ipmitool -H trial077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T02:29:36.507 DEBUG:teuthology.orchestra.console.trial093:check power output: Chassis Power is off
2026-02-13T02:29:36.507 INFO:teuthology.orchestra.console.trial093:Power off completed
2026-02-13T02:29:36.531 DEBUG:teuthology.orchestra.console.trial115:check power output: Chassis Power is off
2026-02-13T02:29:36.531 INFO:teuthology.orchestra.console.trial115:Power off completed
2026-02-13T02:29:36.549 DEBUG:teuthology.orchestra.console.trial077:check power output: Chassis Power is off
2026-02-13T02:29:36.555 INFO:teuthology.orchestra.console.trial077:Power off completed
2026-02-13T02:29:36.795 INFO:teuthology.lock.ops:unlocked: trial093.front.sepia.ceph.com
2026-02-13T02:29:36.806 INFO:teuthology.lock.ops:unlocked: trial115.front.sepia.ceph.com
2026-02-13T02:29:36.810 INFO:teuthology.lock.ops:unlocked: trial077.front.sepia.ceph.com