2026-02-14T01:35:30.795 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71
2026-02-14T01:35:30.795 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:35:30.870 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:35:31.131 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x795a71794980>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x795a7119b4c0>, signals=[15])
2026-02-14T01:35:31.131 INFO:teuthology.lock.ops:Start node 'trial129.front.sepia.ceph.com' reimaging
2026-02-14T01:35:31.131 INFO:teuthology.lock.ops:Updating [trial129.front.sepia.ceph.com]: reset os type and version on server
2026-02-14T01:35:31.131 INFO:teuthology.lock.ops:Updating trial129.front.sepia.ceph.com on lock server
2026-02-14T01:35:31.189 INFO:teuthology.lock.ops:Start node 'trial149.front.sepia.ceph.com' reimaging
2026-02-14T01:35:31.189 INFO:teuthology.lock.ops:Updating [trial149.front.sepia.ceph.com]: reset os type and version on server
2026-02-14T01:35:31.189 INFO:teuthology.lock.ops:Updating trial149.front.sepia.ceph.com on lock server
2026-02-14T01:35:31.246 INFO:teuthology.lock.ops:Start node 'trial150.front.sepia.ceph.com' reimaging
2026-02-14T01:35:31.246 INFO:teuthology.lock.ops:Updating [trial150.front.sepia.ceph.com]: reset os type and version on server
2026-02-14T01:35:31.246 INFO:teuthology.lock.ops:Updating trial150.front.sepia.ceph.com on lock server
2026-02-14T01:35:31.479 DEBUG:teuthology.provision.fog.trial149:Requesting image trial_centos_9.stream (ID 3)
2026-02-14T01:35:31.479 DEBUG:teuthology.provision.fog.trial129:Requesting image trial_centos_9.stream (ID 3)
2026-02-14T01:35:31.532 INFO:teuthology.provision.fog.trial149:Scheduling deploy of centos 9.stream
2026-02-14T01:35:31.534 INFO:teuthology.provision.fog.trial129:Scheduling deploy of centos 9.stream
2026-02-14T01:35:31.537 DEBUG:teuthology.provision.fog.trial150:Requesting image trial_centos_9.stream (ID 3)
2026-02-14T01:35:31.591 INFO:teuthology.provision.fog.trial150:Scheduling deploy of centos 9.stream
2026-02-14T01:35:33.035 INFO:teuthology.orchestra.console.trial149:Power off
2026-02-14T01:35:33.035 DEBUG:teuthology.orchestra.console.trial149:pexpect command: ipmitool -H trial149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T01:35:33.045 INFO:teuthology.orchestra.console.trial129:Power off
2026-02-14T01:35:33.045 DEBUG:teuthology.orchestra.console.trial129:pexpect command: ipmitool -H trial129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T01:35:33.055 INFO:teuthology.orchestra.console.trial150:Power off
2026-02-14T01:35:33.055 DEBUG:teuthology.orchestra.console.trial150:pexpect command: ipmitool -H trial150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T01:35:33.084 DEBUG:teuthology.orchestra.console.trial149:power off output: Chassis Power Control: Down/Off
2026-02-14T01:35:33.096 DEBUG:teuthology.orchestra.console.trial129:power off output: Chassis Power Control: Down/Off
2026-02-14T01:35:33.117 DEBUG:teuthology.orchestra.console.trial150:power off output: Chassis Power Control: Down/Off
2026-02-14T01:35:37.088 DEBUG:teuthology.orchestra.console.trial149:pexpect command: ipmitool -H trial149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:35:37.096 DEBUG:teuthology.orchestra.console.trial129:pexpect command: ipmitool -H trial129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:35:37.119 DEBUG:teuthology.orchestra.console.trial150:pexpect command: ipmitool -H trial150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:35:37.196 DEBUG:teuthology.orchestra.console.trial149:check power output: Chassis Power is off
2026-02-14T01:35:37.197 INFO:teuthology.orchestra.console.trial149:Power off completed
2026-02-14T01:35:37.204 DEBUG:teuthology.orchestra.console.trial129:check power output: Chassis Power is off
2026-02-14T01:35:37.204 INFO:teuthology.orchestra.console.trial129:Power off completed
2026-02-14T01:35:37.225 DEBUG:teuthology.orchestra.console.trial150:check power output: Chassis Power is off
2026-02-14T01:35:37.226 INFO:teuthology.orchestra.console.trial150:Power off completed
2026-02-14T01:35:37.298 INFO:teuthology.orchestra.console.trial149:Power on
2026-02-14T01:35:37.298 DEBUG:teuthology.orchestra.console.trial149:pexpect command: ipmitool -H trial149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-14T01:35:37.308 INFO:teuthology.orchestra.console.trial129:Power on
2026-02-14T01:35:37.308 DEBUG:teuthology.orchestra.console.trial129:pexpect command: ipmitool -H trial129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-14T01:35:37.326 INFO:teuthology.orchestra.console.trial150:Power on
2026-02-14T01:35:37.327 DEBUG:teuthology.orchestra.console.trial150:pexpect command: ipmitool -H trial150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-14T01:35:37.339 DEBUG:teuthology.orchestra.console.trial149:power on output: Chassis Power Control: Up/On
2026-02-14T01:35:37.348 DEBUG:teuthology.orchestra.console.trial129:power on output: Chassis Power Control: Up/On
2026-02-14T01:35:37.364 DEBUG:teuthology.orchestra.console.trial150:power on output: Chassis Power Control: Up/On
2026-02-14T01:35:41.343 DEBUG:teuthology.orchestra.console.trial149:pexpect command: ipmitool -H trial149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:35:41.351 DEBUG:teuthology.orchestra.console.trial129:pexpect command: ipmitool -H trial129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:35:41.366 DEBUG:teuthology.orchestra.console.trial150:pexpect command: ipmitool -H trial150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:35:41.453 DEBUG:teuthology.orchestra.console.trial149:check power output: Chassis Power is off
2026-02-14T01:35:41.460 DEBUG:teuthology.orchestra.console.trial129:check power output: Chassis Power is on
2026-02-14T01:35:41.461 INFO:teuthology.orchestra.console.trial129:Power on completed
2026-02-14T01:35:41.475 DEBUG:teuthology.orchestra.console.trial150:check power output: Chassis Power is on
2026-02-14T01:35:41.475 INFO:teuthology.orchestra.console.trial150:Power on completed
2026-02-14T01:35:41.562 INFO:teuthology.provision.fog.trial129:Waiting for deploy to finish
2026-02-14T01:35:41.577 INFO:teuthology.provision.fog.trial150:Waiting for deploy to finish
2026-02-14T01:35:45.458 DEBUG:teuthology.orchestra.console.trial149:pexpect command: ipmitool -H trial149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:35:45.567 DEBUG:teuthology.orchestra.console.trial149:check power output: Chassis Power is on
2026-02-14T01:35:45.568 INFO:teuthology.orchestra.console.trial149:Power on completed
2026-02-14T01:35:45.668 INFO:teuthology.provision.fog.trial149:Waiting for deploy to finish
2026-02-14T01:37:28.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:37:28.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:37:29.054 ERROR:teuthology.orchestra.connection:Error authenticating with trial150.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2026-02-14T01:37:31.997 WARNING:teuthology.provision.fog.trial129:[Errno None] Unable to connect to port 22 on 10.20.193.129
2026-02-14T01:37:32.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:37:35.901 WARNING:teuthology.provision.fog.trial149:[Errno None] Unable to connect to port 22 on 10.20.193.149
2026-02-14T01:37:37.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:37:41.086 WARNING:teuthology.provision.fog.trial129:[Errno None] Unable to connect to port 22 on 10.20.193.129
2026-02-14T01:37:41.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:37:44.989 WARNING:teuthology.provision.fog.trial149:[Errno None] Unable to connect to port 22 on 10.20.193.149
2026-02-14T01:37:47.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:37:50.173 WARNING:teuthology.provision.fog.trial129:[Errno None] Unable to connect to port 22 on 10.20.193.129
2026-02-14T01:37:50.990 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:37:54.077 WARNING:teuthology.provision.fog.trial149:[Errno None] Unable to connect to port 22 on 10.20.193.149
2026-02-14T01:37:56.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:37:59.261 WARNING:teuthology.provision.fog.trial129:[Errno None] Unable to connect to port 22 on 10.20.193.129
2026-02-14T01:38:00.078 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:38:03.165 WARNING:teuthology.provision.fog.trial149:[Errno None] Unable to connect to port 22 on 10.20.193.149
2026-02-14T01:38:05.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:38:08.349 WARNING:teuthology.provision.fog.trial129:[Errno None] Unable to connect to port 22 on 10.20.193.129
2026-02-14T01:38:09.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:38:12.253 WARNING:teuthology.provision.fog.trial149:[Errno None] Unable to connect to port 22 on 10.20.193.149
2026-02-14T01:38:14.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:38:18.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:38:33.056 WARNING:teuthology.provision.fog.trial150:timed out
2026-02-14T01:38:39.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:38:44.293 ERROR:teuthology.orchestra.connection:Error authenticating with trial150.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-14T01:38:48.409 ERROR:teuthology.orchestra.connection:Error authenticating with trial150.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!)
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/connection.py", line 111, in connect
    ssh.connect(**connect_args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 485, in connect
    self._auth(
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 818, in _auth
    raise saved_exception
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/client.py", line 794, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/paramiko/transport.py", line 1664, in auth_publickey
    raise SSHException("No existing session")
paramiko.ssh_exception.SSHException: No existing session
2026-02-14T01:38:50.264 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-14T01:38:50.264 DEBUG:teuthology.orchestra.run.trial129:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-14T01:38:50.512 INFO:teuthology.provision.fog.trial129:Node is ready
2026-02-14T01:38:50.512 DEBUG:teuthology.orchestra.run.trial129:> hostname
2026-02-14T01:38:50.569 INFO:teuthology.orchestra.run.trial129.stdout:trial129.front.sepia.ceph.com
2026-02-14T01:38:50.569 DEBUG:teuthology.orchestra.run.trial129:> grep trial129.front.sepia.ceph.com /etc/hosts
2026-02-14T01:38:50.626 INFO:teuthology.orchestra.run.trial129.stdout:10.20.193.129 trial129.front.sepia.ceph.com trial129
2026-02-14T01:38:50.627 DEBUG:teuthology.orchestra.run.trial129:> sudo hostname trial129
2026-02-14T01:38:50.695 DEBUG:teuthology.orchestra.run.trial129:> sudo sed -i -e 's/trial129.front.sepia.ceph.com/trial129/g' /etc/hosts
2026-02-14T01:38:50.764 DEBUG:teuthology.orchestra.run.trial129:> sudo sed -i -e 's/10.20.193.129/10.20.193.129/g' /etc/hosts
2026-02-14T01:38:50.829 DEBUG:teuthology.orchestra.run.trial129:> sudo sed -i -e 's/trial129.front.sepia.ceph.com/trial129/g' /etc/hostname
2026-02-14T01:38:50.895 DEBUG:teuthology.orchestra.run.trial129:> sudo hostname trial129
2026-02-14T01:38:50.961 DEBUG:teuthology.orchestra.run.trial129:> cat /etc/os-release
2026-02-14T01:38:51.018 INFO:teuthology.orchestra.run.trial129.stdout:NAME="CentOS Stream"
2026-02-14T01:38:51.018 INFO:teuthology.orchestra.run.trial129.stdout:VERSION="9"
2026-02-14T01:38:51.018 INFO:teuthology.orchestra.run.trial129.stdout:ID="centos"
2026-02-14T01:38:51.018 INFO:teuthology.orchestra.run.trial129.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:38:51.018 INFO:teuthology.orchestra.run.trial129.stdout:VERSION_ID="9"
2026-02-14T01:38:51.018 INFO:teuthology.orchestra.run.trial129.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:38:51.019 INFO:teuthology.orchestra.run.trial129.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:38:51.019 INFO:teuthology.orchestra.run.trial129.stdout:ANSI_COLOR="0;31"
2026-02-14T01:38:51.019 INFO:teuthology.orchestra.run.trial129.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:38:51.019 INFO:teuthology.orchestra.run.trial129.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:38:51.019 INFO:teuthology.orchestra.run.trial129.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:38:51.019 INFO:teuthology.orchestra.run.trial129.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:38:51.019 INFO:teuthology.orchestra.run.trial129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:38:51.019 INFO:teuthology.orchestra.run.trial129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:38:51.020 INFO:teuthology.provision.fog.trial129:Deploy complete!
2026-02-14T01:38:51.021 DEBUG:teuthology.parallel:result is None
2026-02-14T01:38:54.364 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-14T01:38:54.365 DEBUG:teuthology.orchestra.run.trial149:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-14T01:38:54.518 INFO:teuthology.provision.fog.trial149:Node is ready
2026-02-14T01:38:54.518 DEBUG:teuthology.orchestra.run.trial149:> hostname
2026-02-14T01:38:54.573 INFO:teuthology.orchestra.run.trial149.stdout:trial149.front.sepia.ceph.com
2026-02-14T01:38:54.574 DEBUG:teuthology.orchestra.run.trial149:> grep trial149.front.sepia.ceph.com /etc/hosts
2026-02-14T01:38:54.630 INFO:teuthology.orchestra.run.trial149.stdout:10.20.193.149 trial149.front.sepia.ceph.com trial149
2026-02-14T01:38:54.630 DEBUG:teuthology.orchestra.run.trial149:> sudo hostname trial149
2026-02-14T01:38:54.700 DEBUG:teuthology.orchestra.run.trial149:> sudo sed -i -e 's/trial149.front.sepia.ceph.com/trial149/g' /etc/hosts
2026-02-14T01:38:54.766 DEBUG:teuthology.orchestra.run.trial149:> sudo sed -i -e 's/10.20.193.149/10.20.193.149/g' /etc/hosts
2026-02-14T01:38:54.831 DEBUG:teuthology.orchestra.run.trial149:> sudo sed -i -e 's/trial149.front.sepia.ceph.com/trial149/g' /etc/hostname
2026-02-14T01:38:54.896 DEBUG:teuthology.orchestra.run.trial149:> sudo hostname trial149
2026-02-14T01:38:54.960 DEBUG:teuthology.orchestra.run.trial149:> cat /etc/os-release
2026-02-14T01:38:55.017 INFO:teuthology.orchestra.run.trial149.stdout:NAME="CentOS Stream"
2026-02-14T01:38:55.017 INFO:teuthology.orchestra.run.trial149.stdout:VERSION="9"
2026-02-14T01:38:55.017 INFO:teuthology.orchestra.run.trial149.stdout:ID="centos"
2026-02-14T01:38:55.017 INFO:teuthology.orchestra.run.trial149.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:38:55.017 INFO:teuthology.orchestra.run.trial149.stdout:VERSION_ID="9"
2026-02-14T01:38:55.017 INFO:teuthology.orchestra.run.trial149.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:38:55.017 INFO:teuthology.orchestra.run.trial149.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:38:55.017 INFO:teuthology.orchestra.run.trial149.stdout:ANSI_COLOR="0;31"
2026-02-14T01:38:55.017 INFO:teuthology.orchestra.run.trial149.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:38:55.017 INFO:teuthology.orchestra.run.trial149.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:38:55.017 INFO:teuthology.orchestra.run.trial149.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:38:55.018 INFO:teuthology.orchestra.run.trial149.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:38:55.018 INFO:teuthology.orchestra.run.trial149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:38:55.018 INFO:teuthology.orchestra.run.trial149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:38:55.018 INFO:teuthology.provision.fog.trial149:Deploy complete!
2026-02-14T01:38:55.019 DEBUG:teuthology.parallel:result is None
2026-02-14T01:38:55.508 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-14T01:38:55.508 DEBUG:teuthology.orchestra.run.trial150:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-14T01:38:55.678 INFO:teuthology.provision.fog.trial150:Node is ready
2026-02-14T01:38:55.678 DEBUG:teuthology.orchestra.run.trial150:> hostname
2026-02-14T01:38:55.733 INFO:teuthology.orchestra.run.trial150.stdout:trial150.front.sepia.ceph.com
2026-02-14T01:38:55.734 DEBUG:teuthology.orchestra.run.trial150:> grep trial150.front.sepia.ceph.com /etc/hosts
2026-02-14T01:38:55.789 INFO:teuthology.orchestra.run.trial150.stdout:10.20.193.150 trial150.front.sepia.ceph.com trial150
2026-02-14T01:38:55.790 DEBUG:teuthology.orchestra.run.trial150:> sudo hostname trial150
2026-02-14T01:38:55.858 DEBUG:teuthology.orchestra.run.trial150:> sudo sed -i -e 's/trial150.front.sepia.ceph.com/trial150/g' /etc/hosts
2026-02-14T01:38:55.923 DEBUG:teuthology.orchestra.run.trial150:> sudo sed -i -e 's/10.20.193.150/10.20.193.150/g' /etc/hosts
2026-02-14T01:38:55.988 DEBUG:teuthology.orchestra.run.trial150:> sudo sed -i -e 's/trial150.front.sepia.ceph.com/trial150/g' /etc/hostname
2026-02-14T01:38:56.052 DEBUG:teuthology.orchestra.run.trial150:> sudo hostname trial150
2026-02-14T01:38:56.116 DEBUG:teuthology.orchestra.run.trial150:> cat /etc/os-release
2026-02-14T01:38:56.171 INFO:teuthology.orchestra.run.trial150.stdout:NAME="CentOS Stream"
2026-02-14T01:38:56.171 INFO:teuthology.orchestra.run.trial150.stdout:VERSION="9"
2026-02-14T01:38:56.171 INFO:teuthology.orchestra.run.trial150.stdout:ID="centos"
2026-02-14T01:38:56.171 INFO:teuthology.orchestra.run.trial150.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:38:56.171 INFO:teuthology.orchestra.run.trial150.stdout:VERSION_ID="9"
2026-02-14T01:38:56.171 INFO:teuthology.orchestra.run.trial150.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:38:56.171 INFO:teuthology.orchestra.run.trial150.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:38:56.172 INFO:teuthology.orchestra.run.trial150.stdout:ANSI_COLOR="0;31"
2026-02-14T01:38:56.172 INFO:teuthology.orchestra.run.trial150.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:38:56.172 INFO:teuthology.orchestra.run.trial150.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:38:56.172 INFO:teuthology.orchestra.run.trial150.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:38:56.172 INFO:teuthology.orchestra.run.trial150.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:38:56.172 INFO:teuthology.orchestra.run.trial150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:38:56.172 INFO:teuthology.orchestra.run.trial150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:38:56.172 INFO:teuthology.provision.fog.trial150:Deploy complete!
2026-02-14T01:38:56.172 DEBUG:teuthology.parallel:result is None
2026-02-14T01:38:56.682 INFO:teuthology.lock.ops:Checking trial129.front.sepia.ceph.com
2026-02-14T01:38:56.682 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-14T01:38:56.746 INFO:teuthology.lock.ops:Checking trial149.front.sepia.ceph.com
2026-02-14T01:38:56.746 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-14T01:38:56.807 INFO:teuthology.lock.ops:Checking trial150.front.sepia.ceph.com
2026-02-14T01:38:56.807 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-14T01:38:56.868 INFO:teuthology.lock.ops:Updating [trial129.front.sepia.ceph.com]: set os type and version on server
2026-02-14T01:38:56.868 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial129.front.sepia.ceph.com'
2026-02-14T01:38:56.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:38:56.933 DEBUG:teuthology.orchestra.run.trial129:> true
2026-02-14T01:38:57.009 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial129.front.sepia.ceph.com'
2026-02-14T01:38:57.009 DEBUG:teuthology.orchestra.run.trial129:> uname -m
2026-02-14T01:38:57.065 INFO:teuthology.orchestra.run.trial129.stdout:x86_64
2026-02-14T01:38:57.065 DEBUG:teuthology.orchestra.run.trial129:> cat /etc/os-release
2026-02-14T01:38:57.122 INFO:teuthology.orchestra.run.trial129.stdout:NAME="CentOS Stream"
2026-02-14T01:38:57.122 INFO:teuthology.orchestra.run.trial129.stdout:VERSION="9"
2026-02-14T01:38:57.122 INFO:teuthology.orchestra.run.trial129.stdout:ID="centos"
2026-02-14T01:38:57.122 INFO:teuthology.orchestra.run.trial129.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:38:57.122 INFO:teuthology.orchestra.run.trial129.stdout:VERSION_ID="9"
2026-02-14T01:38:57.122 INFO:teuthology.orchestra.run.trial129.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:38:57.122 INFO:teuthology.orchestra.run.trial129.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:38:57.122 INFO:teuthology.orchestra.run.trial129.stdout:ANSI_COLOR="0;31"
2026-02-14T01:38:57.122 INFO:teuthology.orchestra.run.trial129.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:38:57.122 INFO:teuthology.orchestra.run.trial129.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:38:57.122 INFO:teuthology.orchestra.run.trial129.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:38:57.123 INFO:teuthology.orchestra.run.trial129.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:38:57.123 INFO:teuthology.orchestra.run.trial129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:38:57.123 INFO:teuthology.orchestra.run.trial129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:38:57.123 INFO:teuthology.lock.ops:Updating trial129.front.sepia.ceph.com on lock server
2026-02-14T01:38:57.225 INFO:teuthology.lock.ops:Updating [trial149.front.sepia.ceph.com]: set os type and version on server
2026-02-14T01:38:57.225 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial149.front.sepia.ceph.com'
2026-02-14T01:38:57.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:38:57.290 DEBUG:teuthology.orchestra.run.trial149:> true
2026-02-14T01:38:57.359 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial149.front.sepia.ceph.com'
2026-02-14T01:38:57.359 DEBUG:teuthology.orchestra.run.trial149:> uname -m
2026-02-14T01:38:57.415 INFO:teuthology.orchestra.run.trial149.stdout:x86_64
2026-02-14T01:38:57.415 DEBUG:teuthology.orchestra.run.trial149:> cat /etc/os-release
2026-02-14T01:38:57.471 INFO:teuthology.orchestra.run.trial149.stdout:NAME="CentOS Stream"
2026-02-14T01:38:57.472 INFO:teuthology.orchestra.run.trial149.stdout:VERSION="9"
2026-02-14T01:38:57.472 INFO:teuthology.orchestra.run.trial149.stdout:ID="centos"
2026-02-14T01:38:57.472 INFO:teuthology.orchestra.run.trial149.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:38:57.472 INFO:teuthology.orchestra.run.trial149.stdout:VERSION_ID="9"
2026-02-14T01:38:57.472 INFO:teuthology.orchestra.run.trial149.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:38:57.472 INFO:teuthology.orchestra.run.trial149.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:38:57.472 INFO:teuthology.orchestra.run.trial149.stdout:ANSI_COLOR="0;31"
2026-02-14T01:38:57.473 INFO:teuthology.orchestra.run.trial149.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:38:57.473 INFO:teuthology.orchestra.run.trial149.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:38:57.473 INFO:teuthology.orchestra.run.trial149.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:38:57.473 INFO:teuthology.orchestra.run.trial149.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:38:57.473 INFO:teuthology.orchestra.run.trial149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:38:57.473 INFO:teuthology.orchestra.run.trial149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:38:57.474 INFO:teuthology.lock.ops:Updating trial149.front.sepia.ceph.com on lock server
2026-02-14T01:38:57.559 INFO:teuthology.lock.ops:Updating [trial150.front.sepia.ceph.com]: set os type and version on server
2026-02-14T01:38:57.559 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial150.front.sepia.ceph.com'
2026-02-14T01:38:57.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-14T01:38:57.623 DEBUG:teuthology.orchestra.run.trial150:> true
2026-02-14T01:38:57.698 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial150.front.sepia.ceph.com'
2026-02-14T01:38:57.698 DEBUG:teuthology.orchestra.run.trial150:> uname -m
2026-02-14T01:38:57.754 INFO:teuthology.orchestra.run.trial150.stdout:x86_64
2026-02-14T01:38:57.755 DEBUG:teuthology.orchestra.run.trial150:> cat /etc/os-release
2026-02-14T01:38:57.810 INFO:teuthology.orchestra.run.trial150.stdout:NAME="CentOS Stream"
2026-02-14T01:38:57.810 INFO:teuthology.orchestra.run.trial150.stdout:VERSION="9"
2026-02-14T01:38:57.810 INFO:teuthology.orchestra.run.trial150.stdout:ID="centos"
2026-02-14T01:38:57.810 INFO:teuthology.orchestra.run.trial150.stdout:ID_LIKE="rhel fedora"
2026-02-14T01:38:57.810 INFO:teuthology.orchestra.run.trial150.stdout:VERSION_ID="9"
2026-02-14T01:38:57.810 INFO:teuthology.orchestra.run.trial150.stdout:PLATFORM_ID="platform:el9"
2026-02-14T01:38:57.810 INFO:teuthology.orchestra.run.trial150.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-14T01:38:57.810 INFO:teuthology.orchestra.run.trial150.stdout:ANSI_COLOR="0;31"
2026-02-14T01:38:57.810 INFO:teuthology.orchestra.run.trial150.stdout:LOGO="fedora-logo-icon"
2026-02-14T01:38:57.810 INFO:teuthology.orchestra.run.trial150.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-14T01:38:57.811 INFO:teuthology.orchestra.run.trial150.stdout:HOME_URL="https://centos.org/"
2026-02-14T01:38:57.811 INFO:teuthology.orchestra.run.trial150.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-14T01:38:57.811 INFO:teuthology.orchestra.run.trial150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-14T01:38:57.811 INFO:teuthology.orchestra.run.trial150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-14T01:38:57.811 INFO:teuthology.lock.ops:Updating trial150.front.sepia.ceph.com on lock server
2026-02-14T01:38:57.904 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:38:57.999 INFO:teuthology.dispatcher.supervisor:Running job 50570
2026-02-14T01:38:57.999 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/bin/teuthology -v --owner scheduled_jcollin@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/jcollin-2026-02-14_00:12:18-fs-wip-jcollin-testing-20260213.113451-tentacle-distro-default-trial/50570 --name jcollin-2026-02-14_00:12:18-fs-wip-jcollin-testing-20260213.113451-tentacle-distro-default-trial --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} -- /home/teuthworker/mnt/teuthology/jcollin-2026-02-14_00:12:18-fs-wip-jcollin-testing-20260213.113451-tentacle-distro-default-trial/50570/orig.config.yaml
2026-02-14T01:38:57.999 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/jcollin-2026-02-14_00:12:18-fs-wip-jcollin-testing-20260213.113451-tentacle-distro-default-trial/50570
2026-02-14T01:38:57.999 INFO:teuthology.dispatcher.supervisor:Job PID: 1597551
2026-02-14T01:38:57.999 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-14T01:40:58.000 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:42:58.125 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:44:58.209 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:46:58.326 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:48:58.419 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:50:58.543 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:52:58.670 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:54:58.790 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:56:58.911 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-14T01:56:59.031 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2026-02-14T01:56:59.210 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-14T01:56:59.949 INFO:teuthology.orchestra.console.trial150:Power off
2026-02-14T01:56:59.949 DEBUG:teuthology.orchestra.console.trial150:pexpect command: ipmitool -H trial150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T01:56:59.961 INFO:teuthology.orchestra.console.trial149:Power off
2026-02-14T01:56:59.961 DEBUG:teuthology.orchestra.console.trial149:pexpect command: ipmitool -H trial149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T01:56:59.970 INFO:teuthology.orchestra.console.trial129:Power off
2026-02-14T01:56:59.970 DEBUG:teuthology.orchestra.console.trial129:pexpect command: ipmitool -H trial129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-14T01:56:59.987 DEBUG:teuthology.orchestra.console.trial150:power off output: Chassis Power Control: Down/Off
2026-02-14T01:56:59.998 DEBUG:teuthology.orchestra.console.trial149:power off output: Chassis Power Control: Down/Off
2026-02-14T01:57:00.008 DEBUG:teuthology.orchestra.console.trial129:power off output: Chassis Power Control: Down/Off
2026-02-14T01:57:03.988 DEBUG:teuthology.orchestra.console.trial150:pexpect command: ipmitool -H trial150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:57:03.998 DEBUG:teuthology.orchestra.console.trial149:pexpect command: ipmitool -H trial149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:57:04.009 DEBUG:teuthology.orchestra.console.trial129:pexpect command: ipmitool -H trial129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:57:04.097 DEBUG:teuthology.orchestra.console.trial150:check power output: Chassis Power is on
2026-02-14T01:57:04.108 DEBUG:teuthology.orchestra.console.trial149:check power output: Chassis Power is on
2026-02-14T01:57:04.118 DEBUG:teuthology.orchestra.console.trial129:check power output: Chassis Power is on
2026-02-14T01:57:08.098 DEBUG:teuthology.orchestra.console.trial150:pexpect command: ipmitool -H trial150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:57:08.108 DEBUG:teuthology.orchestra.console.trial149:pexpect command: ipmitool -H trial149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:57:08.119 DEBUG:teuthology.orchestra.console.trial129:pexpect command: ipmitool -H trial129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-14T01:57:08.207 DEBUG:teuthology.orchestra.console.trial150:check power output: Chassis Power is off
2026-02-14T01:57:08.208 INFO:teuthology.orchestra.console.trial150:Power off completed
2026-02-14T01:57:08.216 DEBUG:teuthology.orchestra.console.trial149:check power output: Chassis Power is off
2026-02-14T01:57:08.216 INFO:teuthology.orchestra.console.trial149:Power off completed
2026-02-14T01:57:08.226 DEBUG:teuthology.orchestra.console.trial129:check power output: Chassis Power is off
2026-02-14T01:57:08.226 INFO:teuthology.orchestra.console.trial129:Power off completed
2026-02-14T01:57:08.502 INFO:teuthology.lock.ops:unlocked: trial150.front.sepia.ceph.com
2026-02-14T01:57:08.513 INFO:teuthology.lock.ops:unlocked: trial129.front.sepia.ceph.com
2026-02-14T01:57:08.516 INFO:teuthology.lock.ops:unlocked: trial149.front.sepia.ceph.com