2026-02-13T03:54:33.845 INFO:root:teuthology version: 1.2.4.dev9+ga2c505dae
2026-02-13T03:54:33.845 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T03:54:33.918 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T03:54:34.111 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x76793acbaf90>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x76793a555300>, signals=[15])
2026-02-13T03:54:34.111 INFO:teuthology.lock.ops:Start node 'trial075.front.sepia.ceph.com' reimaging
2026-02-13T03:54:34.111 INFO:teuthology.lock.ops:Updating [trial075.front.sepia.ceph.com]: reset os type and version on server
2026-02-13T03:54:34.111 INFO:teuthology.lock.ops:Updating trial075.front.sepia.ceph.com on lock server
2026-02-13T03:54:34.168 INFO:teuthology.lock.ops:Start node 'trial109.front.sepia.ceph.com' reimaging
2026-02-13T03:54:34.168 INFO:teuthology.lock.ops:Updating [trial109.front.sepia.ceph.com]: reset os type and version on server
2026-02-13T03:54:34.168 INFO:teuthology.lock.ops:Updating trial109.front.sepia.ceph.com on lock server
2026-02-13T03:54:34.399 DEBUG:teuthology.provision.fog.trial075:Requesting image trial_rocky_10.1 (ID 4)
2026-02-13T03:54:34.409 DEBUG:teuthology.provision.fog.trial109:Requesting image trial_rocky_10.1 (ID 4)
2026-02-13T03:54:34.452 INFO:teuthology.provision.fog.trial075:Scheduling deploy of rocky 10.1
2026-02-13T03:54:34.461 INFO:teuthology.provision.fog.trial109:Scheduling deploy of rocky 10.1
2026-02-13T03:54:35.474 INFO:teuthology.orchestra.console.trial109:Power off
2026-02-13T03:54:35.474 DEBUG:teuthology.orchestra.console.trial109:pexpect command: ipmitool -H trial109.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-13T03:54:35.492 INFO:teuthology.orchestra.console.trial075:Power off
2026-02-13T03:54:35.492 DEBUG:teuthology.orchestra.console.trial075:pexpect command: ipmitool -H trial075.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-13T03:54:35.510 DEBUG:teuthology.orchestra.console.trial109:power off output: Chassis Power Control: Down/Off
2026-02-13T03:54:35.528 DEBUG:teuthology.orchestra.console.trial075:power off output: Chassis Power Control: Down/Off
2026-02-13T03:54:39.514 DEBUG:teuthology.orchestra.console.trial109:pexpect command: ipmitool -H trial109.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T03:54:39.529 DEBUG:teuthology.orchestra.console.trial075:pexpect command: ipmitool -H trial075.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T03:54:39.623 DEBUG:teuthology.orchestra.console.trial109:check power output: Chassis Power is off
2026-02-13T03:54:39.623 INFO:teuthology.orchestra.console.trial109:Power off completed
2026-02-13T03:54:39.637 DEBUG:teuthology.orchestra.console.trial075:check power output: Chassis Power is off
2026-02-13T03:54:39.637 INFO:teuthology.orchestra.console.trial075:Power off completed
2026-02-13T03:54:39.725 INFO:teuthology.orchestra.console.trial109:Power on
2026-02-13T03:54:39.725 DEBUG:teuthology.orchestra.console.trial109:pexpect command: ipmitool -H trial109.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-13T03:54:39.738 INFO:teuthology.orchestra.console.trial075:Power on
2026-02-13T03:54:39.739 DEBUG:teuthology.orchestra.console.trial075:pexpect command: ipmitool -H trial075.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-13T03:54:39.763 DEBUG:teuthology.orchestra.console.trial109:power on output: Chassis Power Control: Up/On
2026-02-13T03:54:39.776 DEBUG:teuthology.orchestra.console.trial075:power on output: Chassis Power Control: Up/On
2026-02-13T03:54:43.768 DEBUG:teuthology.orchestra.console.trial109:pexpect command: ipmitool -H trial109.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T03:54:43.777 DEBUG:teuthology.orchestra.console.trial075:pexpect command: ipmitool -H trial075.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T03:54:43.878 DEBUG:teuthology.orchestra.console.trial109:check power output: Chassis Power is on
2026-02-13T03:54:43.878 INFO:teuthology.orchestra.console.trial109:Power on completed
2026-02-13T03:54:43.885 DEBUG:teuthology.orchestra.console.trial075:check power output: Chassis Power is on
2026-02-13T03:54:43.885 INFO:teuthology.orchestra.console.trial075:Power on completed
2026-02-13T03:54:43.980 INFO:teuthology.provision.fog.trial109:Waiting for deploy to finish
2026-02-13T03:54:43.986 INFO:teuthology.provision.fog.trial075:Waiting for deploy to finish
2026-02-13T03:56:31.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial109.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T03:56:31.309 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial075.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T03:56:34.397 WARNING:teuthology.provision.fog.trial109:[Errno None] Unable to connect to port 22 on 10.20.193.109
2026-02-13T03:56:34.397 WARNING:teuthology.provision.fog.trial075:[Errno None] Unable to connect to port 22 on 10.20.193.75
2026-02-13T03:56:40.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial109.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T03:56:40.406 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial075.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T03:56:43.485 WARNING:teuthology.provision.fog.trial109:[Errno None] Unable to connect to port 22 on 10.20.193.109
2026-02-13T03:56:43.485 WARNING:teuthology.provision.fog.trial075:[Errno None] Unable to connect to port 22 on 10.20.193.75
2026-02-13T03:56:49.491 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial109.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T03:56:49.493 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial075.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T03:56:52.573 WARNING:teuthology.provision.fog.trial109:[Errno None] Unable to connect to port 22 on 10.20.193.109
2026-02-13T03:56:52.573 WARNING:teuthology.provision.fog.trial075:[Errno None] Unable to connect to port 22 on 10.20.193.75
2026-02-13T03:56:58.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial109.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T03:56:58.582 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial075.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T03:57:01.661 WARNING:teuthology.provision.fog.trial109:[Errno None] Unable to connect to port 22 on 10.20.193.109
2026-02-13T03:57:01.661 WARNING:teuthology.provision.fog.trial075:[Errno None] Unable to connect to port 22 on 10.20.193.75
2026-02-13T03:57:07.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial109.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T03:57:07.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial075.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T03:57:10.749 WARNING:teuthology.provision.fog.trial075:[Errno None] Unable to connect to port 22 on 10.20.193.75
2026-02-13T03:57:16.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial075.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T03:57:43.557 ERROR:teuthology.orchestra.connection:Error authenticating with trial109.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-13T03:57:47.626 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-13T03:57:47.626 DEBUG:teuthology.orchestra.run.trial109:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-13T03:57:52.436 INFO:teuthology.provision.fog.trial109:Node is ready
2026-02-13T03:57:52.436 DEBUG:teuthology.orchestra.run.trial109:> hostname
2026-02-13T03:57:52.453 INFO:teuthology.orchestra.run.trial109.stdout:trial109.front.sepia.ceph.com
2026-02-13T03:57:52.453 DEBUG:teuthology.orchestra.run.trial109:> grep trial109.front.sepia.ceph.com /etc/hosts
2026-02-13T03:57:52.509 INFO:teuthology.orchestra.run.trial109.stdout:10.20.193.109 trial109.front.sepia.ceph.com trial109
2026-02-13T03:57:52.509 DEBUG:teuthology.orchestra.run.trial109:> sudo hostname trial109
2026-02-13T03:57:52.578 DEBUG:teuthology.orchestra.run.trial109:> sudo sed -i -e 's/trial109.front.sepia.ceph.com/trial109/g' /etc/hosts
2026-02-13T03:57:52.643 DEBUG:teuthology.orchestra.run.trial109:> sudo sed -i -e 's/10.20.193.109/10.20.193.109/g' /etc/hosts
2026-02-13T03:57:52.779 DEBUG:teuthology.orchestra.run.trial109:> sudo sed -i -e 's/trial109.front.sepia.ceph.com/trial109/g' /etc/hostname
2026-02-13T03:57:52.802 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-13T03:57:52.802 DEBUG:teuthology.orchestra.run.trial075:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-13T03:57:52.843 DEBUG:teuthology.orchestra.run.trial109:> sudo hostname trial109
2026-02-13T03:57:52.905 DEBUG:teuthology.orchestra.run.trial109:> cat /etc/os-release
2026-02-13T03:57:52.960 INFO:teuthology.orchestra.run.trial109.stdout:NAME="Rocky Linux"
2026-02-13T03:57:52.960 INFO:teuthology.orchestra.run.trial109.stdout:VERSION="10.1 (Red Quartz)"
2026-02-13T03:57:52.960 INFO:teuthology.orchestra.run.trial109.stdout:ID="rocky"
2026-02-13T03:57:52.960 INFO:teuthology.orchestra.run.trial109.stdout:ID_LIKE="rhel centos fedora"
2026-02-13T03:57:52.960 INFO:teuthology.orchestra.run.trial109.stdout:VERSION_ID="10.1"
2026-02-13T03:57:52.960 INFO:teuthology.orchestra.run.trial109.stdout:PLATFORM_ID="platform:el10"
2026-02-13T03:57:52.961 INFO:teuthology.orchestra.run.trial109.stdout:PRETTY_NAME="Rocky Linux 10.1 (Red Quartz)"
2026-02-13T03:57:52.961 INFO:teuthology.orchestra.run.trial109.stdout:ANSI_COLOR="0;32"
2026-02-13T03:57:52.961 INFO:teuthology.orchestra.run.trial109.stdout:LOGO="fedora-logo-icon"
2026-02-13T03:57:52.961 INFO:teuthology.orchestra.run.trial109.stdout:CPE_NAME="cpe:/o:rocky:rocky:10::baseos"
2026-02-13T03:57:52.961 INFO:teuthology.orchestra.run.trial109.stdout:HOME_URL="https://rockylinux.org/"
2026-02-13T03:57:52.961 INFO:teuthology.orchestra.run.trial109.stdout:VENDOR_NAME="RESF"
2026-02-13T03:57:52.961 INFO:teuthology.orchestra.run.trial109.stdout:VENDOR_URL="https://resf.org/"
2026-02-13T03:57:52.961 INFO:teuthology.orchestra.run.trial109.stdout:BUG_REPORT_URL="https://bugs.rockylinux.org/"
2026-02-13T03:57:52.961 INFO:teuthology.orchestra.run.trial109.stdout:SUPPORT_END="2035-05-31"
2026-02-13T03:57:52.961 INFO:teuthology.orchestra.run.trial109.stdout:ROCKY_SUPPORT_PRODUCT="Rocky-Linux-10"
2026-02-13T03:57:52.961 INFO:teuthology.orchestra.run.trial109.stdout:ROCKY_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T03:57:52.961 INFO:teuthology.orchestra.run.trial109.stdout:REDHAT_SUPPORT_PRODUCT="Rocky Linux"
2026-02-13T03:57:52.962 INFO:teuthology.orchestra.run.trial109.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T03:57:52.962 INFO:teuthology.provision.fog.trial109:Deploy complete!
2026-02-13T03:57:52.963 DEBUG:teuthology.parallel:result is None
2026-02-13T03:57:57.986 INFO:teuthology.provision.fog.trial075:Node is ready
2026-02-13T03:57:57.986 DEBUG:teuthology.orchestra.run.trial075:> hostname
2026-02-13T03:57:58.004 INFO:teuthology.orchestra.run.trial075.stdout:trial075.front.sepia.ceph.com
2026-02-13T03:57:58.004 DEBUG:teuthology.orchestra.run.trial075:> grep trial075.front.sepia.ceph.com /etc/hosts
2026-02-13T03:57:58.062 INFO:teuthology.orchestra.run.trial075.stdout:10.20.193.75 trial075.front.sepia.ceph.com trial075
2026-02-13T03:57:58.062 DEBUG:teuthology.orchestra.run.trial075:> sudo hostname trial075
2026-02-13T03:57:58.135 DEBUG:teuthology.orchestra.run.trial075:> sudo sed -i -e 's/trial075.front.sepia.ceph.com/trial075/g' /etc/hosts
2026-02-13T03:57:58.203 DEBUG:teuthology.orchestra.run.trial075:> sudo sed -i -e 's/10.20.193.75/10.20.193.75/g' /etc/hosts
2026-02-13T03:57:58.270 DEBUG:teuthology.orchestra.run.trial075:> sudo sed -i -e 's/trial075.front.sepia.ceph.com/trial075/g' /etc/hostname
2026-02-13T03:57:58.337 DEBUG:teuthology.orchestra.run.trial075:> sudo hostname trial075
2026-02-13T03:57:58.404 DEBUG:teuthology.orchestra.run.trial075:> cat /etc/os-release
2026-02-13T03:57:58.461 INFO:teuthology.orchestra.run.trial075.stdout:NAME="Rocky Linux"
2026-02-13T03:57:58.461 INFO:teuthology.orchestra.run.trial075.stdout:VERSION="10.1 (Red Quartz)"
2026-02-13T03:57:58.462 INFO:teuthology.orchestra.run.trial075.stdout:ID="rocky"
2026-02-13T03:57:58.462 INFO:teuthology.orchestra.run.trial075.stdout:ID_LIKE="rhel centos fedora"
2026-02-13T03:57:58.462 INFO:teuthology.orchestra.run.trial075.stdout:VERSION_ID="10.1"
2026-02-13T03:57:58.462 INFO:teuthology.orchestra.run.trial075.stdout:PLATFORM_ID="platform:el10"
2026-02-13T03:57:58.462 INFO:teuthology.orchestra.run.trial075.stdout:PRETTY_NAME="Rocky Linux 10.1 (Red Quartz)"
2026-02-13T03:57:58.462 INFO:teuthology.orchestra.run.trial075.stdout:ANSI_COLOR="0;32"
2026-02-13T03:57:58.462 INFO:teuthology.orchestra.run.trial075.stdout:LOGO="fedora-logo-icon"
2026-02-13T03:57:58.462 INFO:teuthology.orchestra.run.trial075.stdout:CPE_NAME="cpe:/o:rocky:rocky:10::baseos"
2026-02-13T03:57:58.462 INFO:teuthology.orchestra.run.trial075.stdout:HOME_URL="https://rockylinux.org/"
2026-02-13T03:57:58.462 INFO:teuthology.orchestra.run.trial075.stdout:VENDOR_NAME="RESF"
2026-02-13T03:57:58.462 INFO:teuthology.orchestra.run.trial075.stdout:VENDOR_URL="https://resf.org/"
2026-02-13T03:57:58.462 INFO:teuthology.orchestra.run.trial075.stdout:BUG_REPORT_URL="https://bugs.rockylinux.org/"
2026-02-13T03:57:58.463 INFO:teuthology.orchestra.run.trial075.stdout:SUPPORT_END="2035-05-31"
2026-02-13T03:57:58.463 INFO:teuthology.orchestra.run.trial075.stdout:ROCKY_SUPPORT_PRODUCT="Rocky-Linux-10"
2026-02-13T03:57:58.463 INFO:teuthology.orchestra.run.trial075.stdout:ROCKY_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T03:57:58.463 INFO:teuthology.orchestra.run.trial075.stdout:REDHAT_SUPPORT_PRODUCT="Rocky Linux"
2026-02-13T03:57:58.463 INFO:teuthology.orchestra.run.trial075.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T03:57:58.463 INFO:teuthology.provision.fog.trial075:Deploy complete!
2026-02-13T03:57:58.464 DEBUG:teuthology.parallel:result is None
2026-02-13T03:57:58.854 INFO:teuthology.lock.ops:Checking trial075.front.sepia.ceph.com
2026-02-13T03:57:58.854 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-13T03:57:58.938 INFO:teuthology.lock.ops:Checking trial109.front.sepia.ceph.com
2026-02-13T03:57:58.938 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-13T03:57:58.998 INFO:teuthology.lock.ops:Updating [trial075.front.sepia.ceph.com]: set os type and version on server
2026-02-13T03:57:58.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial075.front.sepia.ceph.com'
2026-02-13T03:57:58.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial075.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T03:57:59.061 DEBUG:teuthology.orchestra.run.trial075:> true
2026-02-13T03:57:59.136 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial075.front.sepia.ceph.com'
2026-02-13T03:57:59.136 DEBUG:teuthology.orchestra.run.trial075:> uname -m
2026-02-13T03:57:59.194 INFO:teuthology.orchestra.run.trial075.stdout:x86_64
2026-02-13T03:57:59.194 DEBUG:teuthology.orchestra.run.trial075:> cat /etc/os-release
2026-02-13T03:57:59.251 INFO:teuthology.orchestra.run.trial075.stdout:NAME="Rocky Linux"
2026-02-13T03:57:59.251 INFO:teuthology.orchestra.run.trial075.stdout:VERSION="10.1 (Red Quartz)"
2026-02-13T03:57:59.252 INFO:teuthology.orchestra.run.trial075.stdout:ID="rocky"
2026-02-13T03:57:59.252 INFO:teuthology.orchestra.run.trial075.stdout:ID_LIKE="rhel centos fedora"
2026-02-13T03:57:59.252 INFO:teuthology.orchestra.run.trial075.stdout:VERSION_ID="10.1"
2026-02-13T03:57:59.252 INFO:teuthology.orchestra.run.trial075.stdout:PLATFORM_ID="platform:el10"
2026-02-13T03:57:59.252 INFO:teuthology.orchestra.run.trial075.stdout:PRETTY_NAME="Rocky Linux 10.1 (Red Quartz)"
2026-02-13T03:57:59.252 INFO:teuthology.orchestra.run.trial075.stdout:ANSI_COLOR="0;32"
2026-02-13T03:57:59.252 INFO:teuthology.orchestra.run.trial075.stdout:LOGO="fedora-logo-icon"
2026-02-13T03:57:59.252 INFO:teuthology.orchestra.run.trial075.stdout:CPE_NAME="cpe:/o:rocky:rocky:10::baseos"
2026-02-13T03:57:59.252 INFO:teuthology.orchestra.run.trial075.stdout:HOME_URL="https://rockylinux.org/"
2026-02-13T03:57:59.252 INFO:teuthology.orchestra.run.trial075.stdout:VENDOR_NAME="RESF"
2026-02-13T03:57:59.252 INFO:teuthology.orchestra.run.trial075.stdout:VENDOR_URL="https://resf.org/"
2026-02-13T03:57:59.252 INFO:teuthology.orchestra.run.trial075.stdout:BUG_REPORT_URL="https://bugs.rockylinux.org/"
2026-02-13T03:57:59.253 INFO:teuthology.orchestra.run.trial075.stdout:SUPPORT_END="2035-05-31"
2026-02-13T03:57:59.253 INFO:teuthology.orchestra.run.trial075.stdout:ROCKY_SUPPORT_PRODUCT="Rocky-Linux-10"
2026-02-13T03:57:59.253 INFO:teuthology.orchestra.run.trial075.stdout:ROCKY_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T03:57:59.253 INFO:teuthology.orchestra.run.trial075.stdout:REDHAT_SUPPORT_PRODUCT="Rocky Linux"
2026-02-13T03:57:59.253 INFO:teuthology.orchestra.run.trial075.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T03:57:59.254 INFO:teuthology.lock.ops:Updating trial075.front.sepia.ceph.com on lock server
2026-02-13T03:57:59.357 INFO:teuthology.lock.ops:Updating [trial109.front.sepia.ceph.com]: set os type and version on server
2026-02-13T03:57:59.357 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial109.front.sepia.ceph.com'
2026-02-13T03:57:59.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial109.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T03:57:59.428 DEBUG:teuthology.orchestra.run.trial109:> true
2026-02-13T03:57:59.501 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial109.front.sepia.ceph.com'
2026-02-13T03:57:59.501 DEBUG:teuthology.orchestra.run.trial109:> uname -m
2026-02-13T03:57:59.557 INFO:teuthology.orchestra.run.trial109.stdout:x86_64
2026-02-13T03:57:59.558 DEBUG:teuthology.orchestra.run.trial109:> cat /etc/os-release
2026-02-13T03:57:59.615 INFO:teuthology.orchestra.run.trial109.stdout:NAME="Rocky Linux"
2026-02-13T03:57:59.615 INFO:teuthology.orchestra.run.trial109.stdout:VERSION="10.1 (Red Quartz)"
2026-02-13T03:57:59.615 INFO:teuthology.orchestra.run.trial109.stdout:ID="rocky"
2026-02-13T03:57:59.615 INFO:teuthology.orchestra.run.trial109.stdout:ID_LIKE="rhel centos fedora"
2026-02-13T03:57:59.615 INFO:teuthology.orchestra.run.trial109.stdout:VERSION_ID="10.1"
2026-02-13T03:57:59.615 INFO:teuthology.orchestra.run.trial109.stdout:PLATFORM_ID="platform:el10"
2026-02-13T03:57:59.615 INFO:teuthology.orchestra.run.trial109.stdout:PRETTY_NAME="Rocky Linux 10.1 (Red Quartz)"
2026-02-13T03:57:59.615 INFO:teuthology.orchestra.run.trial109.stdout:ANSI_COLOR="0;32"
2026-02-13T03:57:59.616 INFO:teuthology.orchestra.run.trial109.stdout:LOGO="fedora-logo-icon"
2026-02-13T03:57:59.616 INFO:teuthology.orchestra.run.trial109.stdout:CPE_NAME="cpe:/o:rocky:rocky:10::baseos"
2026-02-13T03:57:59.616 INFO:teuthology.orchestra.run.trial109.stdout:HOME_URL="https://rockylinux.org/"
2026-02-13T03:57:59.616 INFO:teuthology.orchestra.run.trial109.stdout:VENDOR_NAME="RESF"
2026-02-13T03:57:59.616 INFO:teuthology.orchestra.run.trial109.stdout:VENDOR_URL="https://resf.org/"
2026-02-13T03:57:59.616 INFO:teuthology.orchestra.run.trial109.stdout:BUG_REPORT_URL="https://bugs.rockylinux.org/"
2026-02-13T03:57:59.616 INFO:teuthology.orchestra.run.trial109.stdout:SUPPORT_END="2035-05-31"
2026-02-13T03:57:59.616 INFO:teuthology.orchestra.run.trial109.stdout:ROCKY_SUPPORT_PRODUCT="Rocky-Linux-10"
2026-02-13T03:57:59.616 INFO:teuthology.orchestra.run.trial109.stdout:ROCKY_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T03:57:59.616 INFO:teuthology.orchestra.run.trial109.stdout:REDHAT_SUPPORT_PRODUCT="Rocky Linux"
2026-02-13T03:57:59.616 INFO:teuthology.orchestra.run.trial109.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="10.1"
2026-02-13T03:57:59.617 INFO:teuthology.lock.ops:Updating trial109.front.sepia.ceph.com on lock server
2026-02-13T03:57:59.720 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T03:57:59.817 INFO:teuthology.dispatcher.supervisor:Running job 48134
2026-02-13T03:57:59.817 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_a2c505dae37458c5537fa44f1320bd04eef25bb1/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48134 --name pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial --description fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/admin} -- /home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48134/orig.config.yaml
2026-02-13T03:57:59.817 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48134
2026-02-13T03:57:59.818 INFO:teuthology.dispatcher.supervisor:Job PID: 1499803
2026-02-13T03:57:59.818 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-13T03:59:59.908 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:02:00.114 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:04:00.323 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:06:00.469 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:08:00.663 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:10:00.800 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:12:00.989 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:14:01.205 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:16:01.426 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:18:01.643 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:20:01.848 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:22:02.011 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:24:02.108 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:26:02.317 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:28:02.514 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:30:02.663 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:32:02.847 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:34:03.021 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:36:03.379 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:38:03.535 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:40:03.724 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:42:03.923 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:44:04.130 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:46:04.274 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:48:04.442 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:50:04.646 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:52:04.774 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:54:04.965 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:56:05.148 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T04:58:05.315 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:00:05.515 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:02:05.711 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:04:05.904 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:06:06.112 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:08:06.293 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:10:06.431 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:12:06.640 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:14:06.856 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:16:06.967 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:18:07.104 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:20:07.317 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:22:07.409 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:24:07.587 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:26:07.801 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:28:07.910 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:30:08.080 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:32:08.264 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:34:08.409 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:36:08.534 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:38:08.687 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:40:08.891 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:42:09.059 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:44:09.267 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:46:09.437 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:48:09.608 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:50:09.798 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:52:09.939 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:54:10.094 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:56:10.285 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T05:58:10.442 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T06:00:10.621 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T06:02:10.815 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T06:02:10.903 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-13T06:02:11.018 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-13T06:02:11.493 INFO:teuthology.orchestra.console.trial075:Power off
2026-02-13T06:02:11.493 DEBUG:teuthology.orchestra.console.trial075:pexpect command: ipmitool -H trial075.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-13T06:02:11.511 INFO:teuthology.orchestra.console.trial109:Power off
2026-02-13T06:02:11.511 DEBUG:teuthology.orchestra.console.trial109:pexpect command: ipmitool -H trial109.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-13T06:02:11.532 DEBUG:teuthology.orchestra.console.trial075:power off output: Chassis Power Control: Down/Off
2026-02-13T06:02:11.548 DEBUG:teuthology.orchestra.console.trial109:power off output: Chassis Power Control: Down/Off
2026-02-13T06:02:15.536 DEBUG:teuthology.orchestra.console.trial075:pexpect command: ipmitool -H trial075.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T06:02:15.549 DEBUG:teuthology.orchestra.console.trial109:pexpect command: ipmitool -H trial109.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T06:02:15.644 DEBUG:teuthology.orchestra.console.trial075:check power output: Chassis Power is on
2026-02-13T06:02:15.659 DEBUG:teuthology.orchestra.console.trial109:check power output: Chassis Power is on
2026-02-13T06:02:19.650 DEBUG:teuthology.orchestra.console.trial075:pexpect command: ipmitool -H trial075.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T06:02:19.660 DEBUG:teuthology.orchestra.console.trial109:pexpect command: ipmitool -H trial109.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T06:02:19.759 DEBUG:teuthology.orchestra.console.trial075:check power output: Chassis Power is off
2026-02-13T06:02:19.759 INFO:teuthology.orchestra.console.trial075:Power off completed
2026-02-13T06:02:19.769 DEBUG:teuthology.orchestra.console.trial109:check power output: Chassis Power is off
2026-02-13T06:02:19.769 INFO:teuthology.orchestra.console.trial109:Power off completed
2026-02-13T06:02:20.007 INFO:teuthology.lock.ops:unlocked: trial075.front.sepia.ceph.com
2026-02-13T06:02:20.020 INFO:teuthology.lock.ops:unlocked: trial109.front.sepia.ceph.com