2026-02-13T20:08:26.549 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71
2026-02-13T20:08:26.549 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:08:26.621 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:08:26.754 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x73dbb50569f0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x73dbb48782c0>, signals=[15])
2026-02-13T20:08:26.754 INFO:teuthology.lock.ops:Start node 'trial009.front.sepia.ceph.com' reimaging
2026-02-13T20:08:26.755 INFO:teuthology.lock.ops:Updating [trial009.front.sepia.ceph.com]: reset os type and version on server
2026-02-13T20:08:26.755 INFO:teuthology.lock.ops:Updating trial009.front.sepia.ceph.com on lock server
2026-02-13T20:08:26.957 DEBUG:teuthology.provision.fog.trial009:Requesting image trial_centos_9.stream (ID 3)
2026-02-13T20:08:27.008 INFO:teuthology.provision.fog.trial009:Scheduling deploy of centos 9.stream
2026-02-13T20:08:28.190 INFO:teuthology.orchestra.console.trial009:Power off
2026-02-13T20:08:28.190 DEBUG:teuthology.orchestra.console.trial009:pexpect command: ipmitool -H trial009.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-13T20:08:28.227 DEBUG:teuthology.orchestra.console.trial009:power off output: Chassis Power Control: Down/Off
2026-02-13T20:08:32.232 DEBUG:teuthology.orchestra.console.trial009:pexpect command: ipmitool -H trial009.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T20:08:32.342 DEBUG:teuthology.orchestra.console.trial009:check power output: Chassis Power is off
2026-02-13T20:08:32.342 INFO:teuthology.orchestra.console.trial009:Power off completed
2026-02-13T20:08:32.443 INFO:teuthology.orchestra.console.trial009:Power on
2026-02-13T20:08:32.443 DEBUG:teuthology.orchestra.console.trial009:pexpect command: ipmitool -H trial009.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-13T20:08:32.482 DEBUG:teuthology.orchestra.console.trial009:power on output: Chassis Power Control: Up/On
2026-02-13T20:08:36.487 DEBUG:teuthology.orchestra.console.trial009:pexpect command: ipmitool -H trial009.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T20:08:36.594 DEBUG:teuthology.orchestra.console.trial009:check power output: Chassis Power is on
2026-02-13T20:08:36.595 INFO:teuthology.orchestra.console.trial009:Power on completed
2026-02-13T20:08:36.695 INFO:teuthology.provision.fog.trial009:Waiting for deploy to finish
2026-02-13T20:10:24.241 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial009.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T20:10:24.352 ERROR:teuthology.orchestra.connection:Error authenticating with trial009.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2026-02-13T20:11:28.355 WARNING:teuthology.provision.fog.trial009:timed out
2026-02-13T20:11:34.362 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial009.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T20:11:37.575 ERROR:teuthology.orchestra.connection:Error authenticating with trial009.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-13T20:11:41.690 ERROR:teuthology.orchestra.connection:Error authenticating with trial009.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-13T20:11:48.799 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-13T20:11:48.800 DEBUG:teuthology.orchestra.run.trial009:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-13T20:11:48.984 INFO:teuthology.provision.fog.trial009:Node is ready
2026-02-13T20:11:48.985 DEBUG:teuthology.orchestra.run.trial009:> hostname
2026-02-13T20:11:49.041 INFO:teuthology.orchestra.run.trial009.stdout:trial009.front.sepia.ceph.com
2026-02-13T20:11:49.042 DEBUG:teuthology.orchestra.run.trial009:> grep trial009.front.sepia.ceph.com /etc/hosts
2026-02-13T20:11:49.098 INFO:teuthology.orchestra.run.trial009.stdout:10.20.193.9 trial009.front.sepia.ceph.com trial009
2026-02-13T20:11:49.099 DEBUG:teuthology.orchestra.run.trial009:> sudo hostname trial009
2026-02-13T20:11:49.168 DEBUG:teuthology.orchestra.run.trial009:> sudo sed -i -e 's/trial009.front.sepia.ceph.com/trial009/g' /etc/hosts
2026-02-13T20:11:49.234 DEBUG:teuthology.orchestra.run.trial009:> sudo sed -i -e 's/10.20.193.9/10.20.193.9/g' /etc/hosts
2026-02-13T20:11:49.299 DEBUG:teuthology.orchestra.run.trial009:> sudo sed -i -e 's/trial009.front.sepia.ceph.com/trial009/g' /etc/hostname
2026-02-13T20:11:49.366 DEBUG:teuthology.orchestra.run.trial009:> sudo hostname trial009
2026-02-13T20:11:49.429 DEBUG:teuthology.orchestra.run.trial009:> cat /etc/os-release
2026-02-13T20:11:49.485 INFO:teuthology.orchestra.run.trial009.stdout:NAME="CentOS Stream"
2026-02-13T20:11:49.485 INFO:teuthology.orchestra.run.trial009.stdout:VERSION="9"
2026-02-13T20:11:49.485 INFO:teuthology.orchestra.run.trial009.stdout:ID="centos"
2026-02-13T20:11:49.485 INFO:teuthology.orchestra.run.trial009.stdout:ID_LIKE="rhel fedora"
2026-02-13T20:11:49.485 INFO:teuthology.orchestra.run.trial009.stdout:VERSION_ID="9"
2026-02-13T20:11:49.485 INFO:teuthology.orchestra.run.trial009.stdout:PLATFORM_ID="platform:el9"
2026-02-13T20:11:49.485 INFO:teuthology.orchestra.run.trial009.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-13T20:11:49.485 INFO:teuthology.orchestra.run.trial009.stdout:ANSI_COLOR="0;31"
2026-02-13T20:11:49.485 INFO:teuthology.orchestra.run.trial009.stdout:LOGO="fedora-logo-icon"
2026-02-13T20:11:49.485 INFO:teuthology.orchestra.run.trial009.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-13T20:11:49.485 INFO:teuthology.orchestra.run.trial009.stdout:HOME_URL="https://centos.org/"
2026-02-13T20:11:49.485 INFO:teuthology.orchestra.run.trial009.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-13T20:11:49.485 INFO:teuthology.orchestra.run.trial009.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-13T20:11:49.485 INFO:teuthology.orchestra.run.trial009.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-13T20:11:49.486 INFO:teuthology.provision.fog.trial009:Deploy complete!
2026-02-13T20:11:49.486 DEBUG:teuthology.parallel:result is None
2026-02-13T20:11:49.709 INFO:teuthology.lock.ops:Checking trial009.front.sepia.ceph.com
2026-02-13T20:11:49.709 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-13T20:11:49.775 INFO:teuthology.lock.ops:Updating [trial009.front.sepia.ceph.com]: set os type and version on server
2026-02-13T20:11:49.775 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial009.front.sepia.ceph.com'
2026-02-13T20:11:49.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial009.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T20:11:49.835 DEBUG:teuthology.orchestra.run.trial009:> true
2026-02-13T20:11:49.908 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial009.front.sepia.ceph.com'
2026-02-13T20:11:49.908 DEBUG:teuthology.orchestra.run.trial009:> uname -m
2026-02-13T20:11:49.963 INFO:teuthology.orchestra.run.trial009.stdout:x86_64
2026-02-13T20:11:49.964 DEBUG:teuthology.orchestra.run.trial009:> cat /etc/os-release
2026-02-13T20:11:50.019 INFO:teuthology.orchestra.run.trial009.stdout:NAME="CentOS Stream"
2026-02-13T20:11:50.020 INFO:teuthology.orchestra.run.trial009.stdout:VERSION="9"
2026-02-13T20:11:50.020 INFO:teuthology.orchestra.run.trial009.stdout:ID="centos"
2026-02-13T20:11:50.020 INFO:teuthology.orchestra.run.trial009.stdout:ID_LIKE="rhel fedora"
2026-02-13T20:11:50.020 INFO:teuthology.orchestra.run.trial009.stdout:VERSION_ID="9"
2026-02-13T20:11:50.020 INFO:teuthology.orchestra.run.trial009.stdout:PLATFORM_ID="platform:el9"
2026-02-13T20:11:50.020 INFO:teuthology.orchestra.run.trial009.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-13T20:11:50.020 INFO:teuthology.orchestra.run.trial009.stdout:ANSI_COLOR="0;31"
2026-02-13T20:11:50.020 INFO:teuthology.orchestra.run.trial009.stdout:LOGO="fedora-logo-icon"
2026-02-13T20:11:50.020 INFO:teuthology.orchestra.run.trial009.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-13T20:11:50.021 INFO:teuthology.orchestra.run.trial009.stdout:HOME_URL="https://centos.org/"
2026-02-13T20:11:50.021 INFO:teuthology.orchestra.run.trial009.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-13T20:11:50.021 INFO:teuthology.orchestra.run.trial009.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-13T20:11:50.021 INFO:teuthology.orchestra.run.trial009.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-13T20:11:50.022 INFO:teuthology.lock.ops:Updating trial009.front.sepia.ceph.com on lock server
2026-02-13T20:11:50.127 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:11:50.220 INFO:teuthology.dispatcher.supervisor:Running job 49613
2026-02-13T20:11:50.220 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/bin/teuthology -v --owner scheduled_ivancich@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/ivancich-2026-02-13_16:37:14-rgw-wip-eric-testing-1-distro-default-trial/49613 --name ivancich-2026-02-13_16:37:14-rgw-wip-eric-testing-1-distro-default-trial --description rgw/lifecycle/{cluster ignore-pg-availability overrides s3tests-branch supported-random-distro$/{centos_latest} tasks/rgw_s3tests} -- /home/teuthworker/mnt/teuthology/ivancich-2026-02-13_16:37:14-rgw-wip-eric-testing-1-distro-default-trial/49613/orig.config.yaml
2026-02-13T20:11:50.220 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/ivancich-2026-02-13_16:37:14-rgw-wip-eric-testing-1-distro-default-trial/49613
2026-02-13T20:11:50.220 INFO:teuthology.dispatcher.supervisor:Job PID: 3805632
2026-02-13T20:11:50.220 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-13T20:13:50.221 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:15:50.320 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:17:50.423 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:19:50.549 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:21:50.810 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:23:50.920 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:25:51.055 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:27:51.187 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:29:51.315 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:31:51.442 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:33:51.579 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:35:51.699 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:37:51.806 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:39:51.932 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:41:52.055 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:43:52.150 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:45:52.279 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:47:52.405 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:49:52.530 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:51:52.624 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:53:52.698 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:55:52.801 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:57:52.920 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T20:59:53.042 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T21:01:53.173 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T21:03:53.298 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T21:03:53.425 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-13T21:03:53.496 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-13T21:03:53.804 INFO:teuthology.orchestra.console.trial009:Power off
2026-02-13T21:03:53.804 DEBUG:teuthology.orchestra.console.trial009:pexpect command: ipmitool -H trial009.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-13T21:03:53.845 DEBUG:teuthology.orchestra.console.trial009:power off output: Chassis Power Control: Down/Off
2026-02-13T21:03:57.846 DEBUG:teuthology.orchestra.console.trial009:pexpect command: ipmitool -H trial009.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T21:03:57.957 DEBUG:teuthology.orchestra.console.trial009:check power output: Chassis Power is on
2026-02-13T21:04:01.958 DEBUG:teuthology.orchestra.console.trial009:pexpect command: ipmitool -H trial009.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T21:04:02.067 DEBUG:teuthology.orchestra.console.trial009:check power output: Chassis Power is off
2026-02-13T21:04:02.067 INFO:teuthology.orchestra.console.trial009:Power off completed
2026-02-13T21:04:02.300 INFO:teuthology.lock.ops:unlocked: trial009.front.sepia.ceph.com