2026-02-19T16:08:45.459 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-19T16:08:45.460 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-19T16:08:45.533 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-19T16:08:45.792 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7a0190bec4a0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7a01905937e0>, signals=[15])
2026-02-19T16:08:45.792 INFO:teuthology.lock.ops:Start node 'trial036.front.sepia.ceph.com' reimaging
2026-02-19T16:08:45.792 INFO:teuthology.lock.ops:Updating [trial036.front.sepia.ceph.com]: reset os type and version on server
2026-02-19T16:08:45.792 INFO:teuthology.lock.ops:Updating trial036.front.sepia.ceph.com on lock server
2026-02-19T16:08:45.851 INFO:teuthology.lock.ops:Start node 'trial163.front.sepia.ceph.com' reimaging
2026-02-19T16:08:45.852 INFO:teuthology.lock.ops:Updating [trial163.front.sepia.ceph.com]: reset os type and version on server
2026-02-19T16:08:45.852 INFO:teuthology.lock.ops:Updating trial163.front.sepia.ceph.com on lock server
2026-02-19T16:08:45.910 INFO:teuthology.lock.ops:Start node 'trial172.front.sepia.ceph.com' reimaging
2026-02-19T16:08:45.910 INFO:teuthology.lock.ops:Updating [trial172.front.sepia.ceph.com]: reset os type and version on server
2026-02-19T16:08:45.910 INFO:teuthology.lock.ops:Updating trial172.front.sepia.ceph.com on lock server
2026-02-19T16:08:46.087 DEBUG:teuthology.provision.fog.trial036:Requesting image trial_centos_9.stream (ID 3)
2026-02-19T16:08:46.151 INFO:teuthology.provision.fog.trial036:Scheduling deploy of centos 9.stream
2026-02-19T16:08:46.151 DEBUG:teuthology.provision.fog.trial163:Requesting image trial_centos_9.stream (ID 3)
2026-02-19T16:08:46.181 DEBUG:teuthology.provision.fog.trial172:Requesting image trial_centos_9.stream (ID 3)
2026-02-19T16:08:46.202 INFO:teuthology.provision.fog.trial163:Scheduling deploy of centos 9.stream
2026-02-19T16:08:46.233 INFO:teuthology.provision.fog.trial172:Scheduling deploy of centos 9.stream
2026-02-19T16:08:47.305 INFO:teuthology.orchestra.console.trial036:Power off
2026-02-19T16:08:47.305 DEBUG:teuthology.orchestra.console.trial036:pexpect command: ipmitool -H trial036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-19T16:08:47.345 DEBUG:teuthology.orchestra.console.trial036:power off output: Chassis Power Control: Down/Off
2026-02-19T16:08:47.470 INFO:teuthology.orchestra.console.trial163:Power off
2026-02-19T16:08:47.470 DEBUG:teuthology.orchestra.console.trial163:pexpect command: ipmitool -H trial163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-19T16:08:47.489 INFO:teuthology.orchestra.console.trial172:Power off
2026-02-19T16:08:47.489 DEBUG:teuthology.orchestra.console.trial172:pexpect command: ipmitool -H trial172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-19T16:08:47.511 DEBUG:teuthology.orchestra.console.trial163:power off output: Chassis Power Control: Down/Off
2026-02-19T16:08:47.532 DEBUG:teuthology.orchestra.console.trial172:power off output: Chassis Power Control: Down/Off
2026-02-19T16:08:51.349 DEBUG:teuthology.orchestra.console.trial036:pexpect command: ipmitool -H trial036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-19T16:08:51.457 DEBUG:teuthology.orchestra.console.trial036:check power output: Chassis Power is off
2026-02-19T16:08:51.457 INFO:teuthology.orchestra.console.trial036:Power off completed
2026-02-19T16:08:51.513 DEBUG:teuthology.orchestra.console.trial163:pexpect command: ipmitool -H trial163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-19T16:08:51.534 DEBUG:teuthology.orchestra.console.trial172:pexpect command: ipmitool -H trial172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-19T16:08:51.559 INFO:teuthology.orchestra.console.trial036:Power on
2026-02-19T16:08:51.559 DEBUG:teuthology.orchestra.console.trial036:pexpect command: ipmitool -H trial036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-19T16:08:51.598 DEBUG:teuthology.orchestra.console.trial036:power on output: Chassis Power Control: Up/On
2026-02-19T16:08:51.621 DEBUG:teuthology.orchestra.console.trial163:check power output: Chassis Power is off
2026-02-19T16:08:51.621 INFO:teuthology.orchestra.console.trial163:Power off completed
2026-02-19T16:08:51.641 DEBUG:teuthology.orchestra.console.trial172:check power output: Chassis Power is off
2026-02-19T16:08:51.641 INFO:teuthology.orchestra.console.trial172:Power off completed
2026-02-19T16:08:51.722 INFO:teuthology.orchestra.console.trial163:Power on
2026-02-19T16:08:51.722 DEBUG:teuthology.orchestra.console.trial163:pexpect command: ipmitool -H trial163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-19T16:08:51.742 INFO:teuthology.orchestra.console.trial172:Power on
2026-02-19T16:08:51.743 DEBUG:teuthology.orchestra.console.trial172:pexpect command: ipmitool -H trial172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-19T16:08:51.767 DEBUG:teuthology.orchestra.console.trial163:power on output: Chassis Power Control: Up/On
2026-02-19T16:08:51.786 DEBUG:teuthology.orchestra.console.trial172:power on output: Chassis Power Control: Up/On
2026-02-19T16:08:55.599 DEBUG:teuthology.orchestra.console.trial036:pexpect command: ipmitool -H trial036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-19T16:08:55.709 DEBUG:teuthology.orchestra.console.trial036:check power output: Chassis Power is on
2026-02-19T16:08:55.709 INFO:teuthology.orchestra.console.trial036:Power on completed
2026-02-19T16:08:55.768 DEBUG:teuthology.orchestra.console.trial163:pexpect command: ipmitool -H trial163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-19T16:08:55.786 DEBUG:teuthology.orchestra.console.trial172:pexpect command: ipmitool -H trial172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-19T16:08:55.810 INFO:teuthology.provision.fog.trial036:Waiting for deploy to finish
2026-02-19T16:08:55.875 DEBUG:teuthology.orchestra.console.trial163:check power output: Chassis Power is on
2026-02-19T16:08:55.875 INFO:teuthology.orchestra.console.trial163:Power on completed
2026-02-19T16:08:55.895 DEBUG:teuthology.orchestra.console.trial172:check power output: Chassis Power is off
2026-02-19T16:08:55.976 INFO:teuthology.provision.fog.trial163:Waiting for deploy to finish
2026-02-19T16:08:59.899 DEBUG:teuthology.orchestra.console.trial172:pexpect command: ipmitool -H trial172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-19T16:09:00.007 DEBUG:teuthology.orchestra.console.trial172:check power output: Chassis Power is on
2026-02-19T16:09:00.008 INFO:teuthology.orchestra.console.trial172:Power on completed
2026-02-19T16:09:00.108 INFO:teuthology.provision.fog.trial172:Waiting for deploy to finish
2026-02-19T16:10:43.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-19T16:10:43.925 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-19T16:10:44.076 ERROR:teuthology.orchestra.connection:Error authenticating with trial163.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2026-02-19T16:10:47.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-19T16:10:50.688 WARNING:teuthology.provision.fog.trial172:[Errno None] Unable to connect to port 22 on 10.20.193.172
2026-02-19T16:10:56.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-19T16:10:57.792 WARNING:teuthology.provision.fog.trial036:[Errno None] Unable to connect to port 22 on 10.20.193.36
2026-02-19T16:10:59.776 WARNING:teuthology.provision.fog.trial172:[Errno None] Unable to connect to port 22 on 10.20.193.172
2026-02-19T16:11:03.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-19T16:11:05.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-19T16:11:06.879 WARNING:teuthology.provision.fog.trial036:[Errno None] Unable to connect to port 22 on 10.20.193.36
2026-02-19T16:11:08.864 WARNING:teuthology.provision.fog.trial172:[Errno None] Unable to connect to port 22 on 10.20.193.172
2026-02-19T16:11:12.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-19T16:11:14.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-19T16:11:15.968 WARNING:teuthology.provision.fog.trial036:[Errno None] Unable to connect to port 22 on 10.20.193.36
2026-02-19T16:11:17.952 WARNING:teuthology.provision.fog.trial172:[Errno None] Unable to connect to port 22 on 10.20.193.172
2026-02-19T16:11:21.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-19T16:11:23.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-19T16:11:25.055 WARNING:teuthology.provision.fog.trial036:[Errno None] Unable to connect to port 22 on 10.20.193.36
2026-02-19T16:11:27.040 WARNING:teuthology.provision.fog.trial172:[Errno None] Unable to connect to port 22 on 10.20.193.172
2026-02-19T16:11:31.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-19T16:11:33.042 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-19T16:11:48.080 WARNING:teuthology.provision.fog.trial163:timed out
2026-02-19T16:11:54.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-19T16:11:58.310 ERROR:teuthology.orchestra.connection:Error authenticating with trial163.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-19T16:12:02.509 ERROR:teuthology.orchestra.connection:Error authenticating with trial163.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-19T16:12:07.052 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-19T16:12:07.052 DEBUG:teuthology.orchestra.run.trial036:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-19T16:12:07.210 INFO:teuthology.provision.fog.trial036:Node is ready
2026-02-19T16:12:07.211 DEBUG:teuthology.orchestra.run.trial036:> hostname
2026-02-19T16:12:07.267 INFO:teuthology.orchestra.run.trial036.stdout:trial036.front.sepia.ceph.com
2026-02-19T16:12:07.267 DEBUG:teuthology.orchestra.run.trial036:> grep trial036.front.sepia.ceph.com /etc/hosts
2026-02-19T16:12:07.324 INFO:teuthology.orchestra.run.trial036.stdout:10.20.193.36 trial036.front.sepia.ceph.com trial036
2026-02-19T16:12:07.325 DEBUG:teuthology.orchestra.run.trial036:> sudo hostname trial036
2026-02-19T16:12:07.393 DEBUG:teuthology.orchestra.run.trial036:> sudo sed -i -e 's/trial036.front.sepia.ceph.com/trial036/g' /etc/hosts
2026-02-19T16:12:07.457 DEBUG:teuthology.orchestra.run.trial036:> sudo sed -i -e 's/10.20.193.36/10.20.193.36/g' /etc/hosts
2026-02-19T16:12:07.523 DEBUG:teuthology.orchestra.run.trial036:> sudo sed -i -e 's/trial036.front.sepia.ceph.com/trial036/g' /etc/hostname
2026-02-19T16:12:07.588 DEBUG:teuthology.orchestra.run.trial036:> sudo hostname trial036
2026-02-19T16:12:07.652 DEBUG:teuthology.orchestra.run.trial036:> cat /etc/os-release
2026-02-19T16:12:07.707 INFO:teuthology.orchestra.run.trial036.stdout:NAME="CentOS Stream"
2026-02-19T16:12:07.708 INFO:teuthology.orchestra.run.trial036.stdout:VERSION="9"
2026-02-19T16:12:07.708 INFO:teuthology.orchestra.run.trial036.stdout:ID="centos"
2026-02-19T16:12:07.708 INFO:teuthology.orchestra.run.trial036.stdout:ID_LIKE="rhel fedora"
2026-02-19T16:12:07.708 INFO:teuthology.orchestra.run.trial036.stdout:VERSION_ID="9"
2026-02-19T16:12:07.708 INFO:teuthology.orchestra.run.trial036.stdout:PLATFORM_ID="platform:el9"
2026-02-19T16:12:07.708 INFO:teuthology.orchestra.run.trial036.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-19T16:12:07.708 INFO:teuthology.orchestra.run.trial036.stdout:ANSI_COLOR="0;31"
2026-02-19T16:12:07.708 INFO:teuthology.orchestra.run.trial036.stdout:LOGO="fedora-logo-icon"
2026-02-19T16:12:07.708 INFO:teuthology.orchestra.run.trial036.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-19T16:12:07.708 INFO:teuthology.orchestra.run.trial036.stdout:HOME_URL="https://centos.org/"
2026-02-19T16:12:07.709 INFO:teuthology.orchestra.run.trial036.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-19T16:12:07.709 INFO:teuthology.orchestra.run.trial036.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-19T16:12:07.709 INFO:teuthology.orchestra.run.trial036.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-19T16:12:07.710 INFO:teuthology.provision.fog.trial036:Deploy complete!
2026-02-19T16:12:07.711 DEBUG:teuthology.parallel:result is None
2026-02-19T16:12:09.053 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-19T16:12:09.053 DEBUG:teuthology.orchestra.run.trial172:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-19T16:12:09.274 INFO:teuthology.provision.fog.trial172:Node is ready
2026-02-19T16:12:09.274 DEBUG:teuthology.orchestra.run.trial172:> hostname
2026-02-19T16:12:09.334 INFO:teuthology.orchestra.run.trial172.stdout:trial172.front.sepia.ceph.com
2026-02-19T16:12:09.334 DEBUG:teuthology.orchestra.run.trial172:> grep trial172.front.sepia.ceph.com /etc/hosts
2026-02-19T16:12:09.391 INFO:teuthology.orchestra.run.trial172.stdout:10.20.193.172 trial172.front.sepia.ceph.com trial172
2026-02-19T16:12:09.392 DEBUG:teuthology.orchestra.run.trial172:> sudo hostname trial172
2026-02-19T16:12:09.462 DEBUG:teuthology.orchestra.run.trial172:> sudo sed -i -e 's/trial172.front.sepia.ceph.com/trial172/g' /etc/hosts
2026-02-19T16:12:09.610 DEBUG:teuthology.orchestra.run.trial172:> sudo sed -i -e 's/10.20.193.172/10.20.193.172/g' /etc/hosts
2026-02-19T16:12:09.621 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-19T16:12:09.621 DEBUG:teuthology.orchestra.run.trial163:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-19T16:12:09.676 DEBUG:teuthology.orchestra.run.trial172:> sudo sed -i -e 's/trial172.front.sepia.ceph.com/trial172/g' /etc/hostname
2026-02-19T16:12:09.741 DEBUG:teuthology.orchestra.run.trial172:> sudo hostname trial172
2026-02-19T16:12:09.778 INFO:teuthology.provision.fog.trial163:Node is ready
2026-02-19T16:12:09.779 DEBUG:teuthology.orchestra.run.trial163:> hostname
2026-02-19T16:12:09.804 DEBUG:teuthology.orchestra.run.trial172:> cat /etc/os-release
2026-02-19T16:12:09.834 INFO:teuthology.orchestra.run.trial163.stdout:trial163.front.sepia.ceph.com
2026-02-19T16:12:09.834 DEBUG:teuthology.orchestra.run.trial163:> grep trial163.front.sepia.ceph.com /etc/hosts
2026-02-19T16:12:09.860 INFO:teuthology.orchestra.run.trial172.stdout:NAME="CentOS Stream"
2026-02-19T16:12:09.860 INFO:teuthology.orchestra.run.trial172.stdout:VERSION="9"
2026-02-19T16:12:09.860 INFO:teuthology.orchestra.run.trial172.stdout:ID="centos"
2026-02-19T16:12:09.860 INFO:teuthology.orchestra.run.trial172.stdout:ID_LIKE="rhel fedora"
2026-02-19T16:12:09.860 INFO:teuthology.orchestra.run.trial172.stdout:VERSION_ID="9"
2026-02-19T16:12:09.860 INFO:teuthology.orchestra.run.trial172.stdout:PLATFORM_ID="platform:el9"
2026-02-19T16:12:09.860 INFO:teuthology.orchestra.run.trial172.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-19T16:12:09.860 INFO:teuthology.orchestra.run.trial172.stdout:ANSI_COLOR="0;31"
2026-02-19T16:12:09.860 INFO:teuthology.orchestra.run.trial172.stdout:LOGO="fedora-logo-icon"
2026-02-19T16:12:09.860 INFO:teuthology.orchestra.run.trial172.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-19T16:12:09.860 INFO:teuthology.orchestra.run.trial172.stdout:HOME_URL="https://centos.org/"
2026-02-19T16:12:09.860 INFO:teuthology.orchestra.run.trial172.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-19T16:12:09.860 INFO:teuthology.orchestra.run.trial172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-19T16:12:09.860 INFO:teuthology.orchestra.run.trial172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-19T16:12:09.860 INFO:teuthology.provision.fog.trial172:Deploy complete!
2026-02-19T16:12:09.861 DEBUG:teuthology.parallel:result is None
2026-02-19T16:12:09.889 INFO:teuthology.orchestra.run.trial163.stdout:10.20.193.163 trial163.front.sepia.ceph.com trial163
2026-02-19T16:12:09.889 DEBUG:teuthology.orchestra.run.trial163:> sudo hostname trial163
2026-02-19T16:12:09.960 DEBUG:teuthology.orchestra.run.trial163:> sudo sed -i -e 's/trial163.front.sepia.ceph.com/trial163/g' /etc/hosts
2026-02-19T16:12:10.026 DEBUG:teuthology.orchestra.run.trial163:> sudo sed -i -e 's/10.20.193.163/10.20.193.163/g' /etc/hosts
2026-02-19T16:12:10.099 DEBUG:teuthology.orchestra.run.trial163:> sudo sed -i -e 's/trial163.front.sepia.ceph.com/trial163/g' /etc/hostname
2026-02-19T16:12:10.167 DEBUG:teuthology.orchestra.run.trial163:> sudo hostname trial163
2026-02-19T16:12:10.231 DEBUG:teuthology.orchestra.run.trial163:> cat /etc/os-release
2026-02-19T16:12:10.287 INFO:teuthology.orchestra.run.trial163.stdout:NAME="CentOS Stream"
2026-02-19T16:12:10.287 INFO:teuthology.orchestra.run.trial163.stdout:VERSION="9"
2026-02-19T16:12:10.287 INFO:teuthology.orchestra.run.trial163.stdout:ID="centos"
2026-02-19T16:12:10.287 INFO:teuthology.orchestra.run.trial163.stdout:ID_LIKE="rhel fedora"
2026-02-19T16:12:10.287 INFO:teuthology.orchestra.run.trial163.stdout:VERSION_ID="9"
2026-02-19T16:12:10.287 INFO:teuthology.orchestra.run.trial163.stdout:PLATFORM_ID="platform:el9"
2026-02-19T16:12:10.287 INFO:teuthology.orchestra.run.trial163.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-19T16:12:10.287 INFO:teuthology.orchestra.run.trial163.stdout:ANSI_COLOR="0;31"
2026-02-19T16:12:10.287 INFO:teuthology.orchestra.run.trial163.stdout:LOGO="fedora-logo-icon"
2026-02-19T16:12:10.287 INFO:teuthology.orchestra.run.trial163.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-19T16:12:10.287 INFO:teuthology.orchestra.run.trial163.stdout:HOME_URL="https://centos.org/"
2026-02-19T16:12:10.287 INFO:teuthology.orchestra.run.trial163.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-19T16:12:10.287 INFO:teuthology.orchestra.run.trial163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-19T16:12:10.287 INFO:teuthology.orchestra.run.trial163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-19T16:12:10.288 INFO:teuthology.provision.fog.trial163:Deploy complete!
2026-02-19T16:12:10.288 DEBUG:teuthology.parallel:result is None
2026-02-19T16:12:10.804 INFO:teuthology.lock.ops:Checking trial036.front.sepia.ceph.com
2026-02-19T16:12:10.804 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-19T16:12:10.908 INFO:teuthology.lock.ops:Checking trial163.front.sepia.ceph.com
2026-02-19T16:12:10.908 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-19T16:12:10.971 INFO:teuthology.lock.ops:Checking trial172.front.sepia.ceph.com
2026-02-19T16:12:10.972 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-19T16:12:11.028 INFO:teuthology.lock.ops:Updating [trial036.front.sepia.ceph.com]: set os type and version on server
2026-02-19T16:12:11.029 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial036.front.sepia.ceph.com'
2026-02-19T16:12:11.029 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial036.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-19T16:12:11.091 DEBUG:teuthology.orchestra.run.trial036:> true
2026-02-19T16:12:11.163 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial036.front.sepia.ceph.com'
2026-02-19T16:12:11.163 DEBUG:teuthology.orchestra.run.trial036:> uname -m
2026-02-19T16:12:11.218 INFO:teuthology.orchestra.run.trial036.stdout:x86_64
2026-02-19T16:12:11.219 DEBUG:teuthology.orchestra.run.trial036:> cat /etc/os-release
2026-02-19T16:12:11.275 INFO:teuthology.orchestra.run.trial036.stdout:NAME="CentOS Stream"
2026-02-19T16:12:11.275 INFO:teuthology.orchestra.run.trial036.stdout:VERSION="9"
2026-02-19T16:12:11.275 INFO:teuthology.orchestra.run.trial036.stdout:ID="centos"
2026-02-19T16:12:11.275 INFO:teuthology.orchestra.run.trial036.stdout:ID_LIKE="rhel fedora"
2026-02-19T16:12:11.275 INFO:teuthology.orchestra.run.trial036.stdout:VERSION_ID="9"
2026-02-19T16:12:11.275 INFO:teuthology.orchestra.run.trial036.stdout:PLATFORM_ID="platform:el9"
2026-02-19T16:12:11.275 INFO:teuthology.orchestra.run.trial036.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-19T16:12:11.275 INFO:teuthology.orchestra.run.trial036.stdout:ANSI_COLOR="0;31"
2026-02-19T16:12:11.275 INFO:teuthology.orchestra.run.trial036.stdout:LOGO="fedora-logo-icon"
2026-02-19T16:12:11.275 INFO:teuthology.orchestra.run.trial036.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-19T16:12:11.275 INFO:teuthology.orchestra.run.trial036.stdout:HOME_URL="https://centos.org/"
2026-02-19T16:12:11.275 INFO:teuthology.orchestra.run.trial036.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-19T16:12:11.275 INFO:teuthology.orchestra.run.trial036.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-19T16:12:11.276 INFO:teuthology.orchestra.run.trial036.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-19T16:12:11.276 INFO:teuthology.lock.ops:Updating trial036.front.sepia.ceph.com on lock server
2026-02-19T16:12:11.358 INFO:teuthology.lock.ops:Updating [trial163.front.sepia.ceph.com]: set os type and version on server
2026-02-19T16:12:11.358 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial163.front.sepia.ceph.com'
2026-02-19T16:12:11.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-19T16:12:11.421 DEBUG:teuthology.orchestra.run.trial163:> true
2026-02-19T16:12:11.497 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial163.front.sepia.ceph.com'
2026-02-19T16:12:11.497 DEBUG:teuthology.orchestra.run.trial163:> uname -m
2026-02-19T16:12:11.552 INFO:teuthology.orchestra.run.trial163.stdout:x86_64
2026-02-19T16:12:11.553 DEBUG:teuthology.orchestra.run.trial163:> cat /etc/os-release
2026-02-19T16:12:11.608 INFO:teuthology.orchestra.run.trial163.stdout:NAME="CentOS Stream"
2026-02-19T16:12:11.608 INFO:teuthology.orchestra.run.trial163.stdout:VERSION="9"
2026-02-19T16:12:11.608 INFO:teuthology.orchestra.run.trial163.stdout:ID="centos"
2026-02-19T16:12:11.608 INFO:teuthology.orchestra.run.trial163.stdout:ID_LIKE="rhel fedora"
2026-02-19T16:12:11.608 INFO:teuthology.orchestra.run.trial163.stdout:VERSION_ID="9"
2026-02-19T16:12:11.608 INFO:teuthology.orchestra.run.trial163.stdout:PLATFORM_ID="platform:el9"
2026-02-19T16:12:11.608 INFO:teuthology.orchestra.run.trial163.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-19T16:12:11.608 INFO:teuthology.orchestra.run.trial163.stdout:ANSI_COLOR="0;31"
2026-02-19T16:12:11.608 INFO:teuthology.orchestra.run.trial163.stdout:LOGO="fedora-logo-icon"
2026-02-19T16:12:11.608 INFO:teuthology.orchestra.run.trial163.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-19T16:12:11.608 INFO:teuthology.orchestra.run.trial163.stdout:HOME_URL="https://centos.org/"
2026-02-19T16:12:11.608 INFO:teuthology.orchestra.run.trial163.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-19T16:12:11.608 INFO:teuthology.orchestra.run.trial163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-19T16:12:11.608 INFO:teuthology.orchestra.run.trial163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-19T16:12:11.608 INFO:teuthology.lock.ops:Updating trial163.front.sepia.ceph.com on lock server
2026-02-19T16:12:11.667 INFO:teuthology.lock.ops:Updating [trial172.front.sepia.ceph.com]: set os type and version on server
2026-02-19T16:12:11.667 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial172.front.sepia.ceph.com'
2026-02-19T16:12:11.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-19T16:12:11.728 DEBUG:teuthology.orchestra.run.trial172:> true
2026-02-19T16:12:11.801 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial172.front.sepia.ceph.com'
2026-02-19T16:12:11.801 DEBUG:teuthology.orchestra.run.trial172:> uname -m
2026-02-19T16:12:11.856 INFO:teuthology.orchestra.run.trial172.stdout:x86_64
2026-02-19T16:12:11.856 DEBUG:teuthology.orchestra.run.trial172:> cat /etc/os-release
2026-02-19T16:12:11.912 INFO:teuthology.orchestra.run.trial172.stdout:NAME="CentOS Stream"
2026-02-19T16:12:11.912 INFO:teuthology.orchestra.run.trial172.stdout:VERSION="9"
2026-02-19T16:12:11.913 INFO:teuthology.orchestra.run.trial172.stdout:ID="centos"
2026-02-19T16:12:11.913 INFO:teuthology.orchestra.run.trial172.stdout:ID_LIKE="rhel fedora"
2026-02-19T16:12:11.913 INFO:teuthology.orchestra.run.trial172.stdout:VERSION_ID="9"
2026-02-19T16:12:11.913 INFO:teuthology.orchestra.run.trial172.stdout:PLATFORM_ID="platform:el9"
2026-02-19T16:12:11.913 INFO:teuthology.orchestra.run.trial172.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-19T16:12:11.913 INFO:teuthology.orchestra.run.trial172.stdout:ANSI_COLOR="0;31"
2026-02-19T16:12:11.913 INFO:teuthology.orchestra.run.trial172.stdout:LOGO="fedora-logo-icon"
2026-02-19T16:12:11.913 INFO:teuthology.orchestra.run.trial172.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-19T16:12:11.913 INFO:teuthology.orchestra.run.trial172.stdout:HOME_URL="https://centos.org/"
2026-02-19T16:12:11.913 INFO:teuthology.orchestra.run.trial172.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-19T16:12:11.913 INFO:teuthology.orchestra.run.trial172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-19T16:12:11.914 INFO:teuthology.orchestra.run.trial172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-19T16:12:11.914 INFO:teuthology.lock.ops:Updating trial172.front.sepia.ceph.com on lock server
2026-02-19T16:12:12.017 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-19T16:12:12.113 INFO:teuthology.dispatcher.supervisor:Running job 59047
2026-02-19T16:12:12.113 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/bin/teuthology -v --owner scheduled_yuriw@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:38:40-rados-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59047 --name yuriw-2026-02-19_15:38:40-rados-wip-yuri-testing-2026-02-18-1820-distro-default-trial --description rados/thrash-old-clients/{0-distro$/{centos_9.stream} 0-size-min-size-overrides/3-size-2-min-size 1-install/reef backoff/peering_and_degraded ceph clusters/{three-plus-one} d-balancer/on mon_election/classic msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} -- /home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:38:40-rados-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59047/orig.config.yaml
2026-02-19T16:12:12.114 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:38:40-rados-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59047
2026-02-19T16:12:12.114 INFO:teuthology.dispatcher.supervisor:Job PID: 3856466
2026-02-19T16:12:12.114 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-19T16:14:12.114 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-19T16:16:12.204 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-19T16:18:12.290 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-19T16:20:12.403 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-19T16:20:12.530 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2026-02-19T16:20:12.714 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-19T16:20:13.454 INFO:teuthology.orchestra.console.trial036:Power off
2026-02-19T16:20:13.455 DEBUG:teuthology.orchestra.console.trial036:pexpect command: ipmitool -H trial036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-19T16:20:13.467 INFO:teuthology.orchestra.console.trial163:Power off
2026-02-19T16:20:13.467 DEBUG:teuthology.orchestra.console.trial163:pexpect command: ipmitool -H trial163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-19T16:20:13.478 INFO:teuthology.orchestra.console.trial172:Power off
2026-02-19T16:20:13.478 DEBUG:teuthology.orchestra.console.trial172:pexpect command: ipmitool -H trial172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-19T16:20:13.494 DEBUG:teuthology.orchestra.console.trial036:power off output: Chassis Power Control: Down/Off
2026-02-19T16:20:13.505 DEBUG:teuthology.orchestra.console.trial163:power off output: Chassis Power Control: Down/Off
2026-02-19T16:20:13.515 DEBUG:teuthology.orchestra.console.trial172:power off output: Chassis Power Control: Down/Off
2026-02-19T16:20:17.495 DEBUG:teuthology.orchestra.console.trial036:pexpect command: ipmitool -H trial036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-19T16:20:17.505 DEBUG:teuthology.orchestra.console.trial163:pexpect command: ipmitool -H trial163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-19T16:20:17.516 DEBUG:teuthology.orchestra.console.trial172:pexpect command: ipmitool -H trial172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-19T16:20:17.604 DEBUG:teuthology.orchestra.console.trial036:check power output: Chassis Power is on
2026-02-19T16:20:17.614 DEBUG:teuthology.orchestra.console.trial163:check power output: Chassis Power is on
2026-02-19T16:20:17.622 DEBUG:teuthology.orchestra.console.trial172:check power output: Chassis Power is on
2026-02-19T16:20:21.605 DEBUG:teuthology.orchestra.console.trial036:pexpect command: ipmitool -H trial036.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-19T16:20:21.615 DEBUG:teuthology.orchestra.console.trial163:pexpect command: ipmitool -H trial163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-19T16:20:21.623 DEBUG:teuthology.orchestra.console.trial172:pexpect command: ipmitool -H trial172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-19T16:20:21.715 DEBUG:teuthology.orchestra.console.trial036:check power output: Chassis Power is off
2026-02-19T16:20:21.715 INFO:teuthology.orchestra.console.trial036:Power off completed
2026-02-19T16:20:21.725 DEBUG:teuthology.orchestra.console.trial163:check power output: Chassis Power is on
2026-02-19T16:20:21.732 DEBUG:teuthology.orchestra.console.trial172:check power output: Chassis Power is off
2026-02-19T16:20:21.732 INFO:teuthology.orchestra.console.trial172:Power off completed
2026-02-19T16:20:21.962 INFO:teuthology.lock.ops:unlocked: trial036.front.sepia.ceph.com
2026-02-19T16:20:21.973 INFO:teuthology.lock.ops:unlocked: trial172.front.sepia.ceph.com
2026-02-19T16:20:25.726 DEBUG:teuthology.orchestra.console.trial163:pexpect command: ipmitool -H trial163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-19T16:20:25.837 DEBUG:teuthology.orchestra.console.trial163:check power output: Chassis Power is off
2026-02-19T16:20:25.838 INFO:teuthology.orchestra.console.trial163:Power off completed
2026-02-19T16:20:26.047 INFO:teuthology.lock.ops:unlocked: trial163.front.sepia.ceph.com