2026-02-12T08:41:18.561 INFO:root:teuthology version: 0.0.1.dev29+gc433f1062
2026-02-12T08:41:18.561 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:41:18.635 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:41:18.830 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7543e8d348f0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7543e8c9a160>, signals=[15])
2026-02-12T08:41:18.830 INFO:teuthology.lock.ops:Start node 'trial158.front.sepia.ceph.com' reimaging
2026-02-12T08:41:18.830 INFO:teuthology.lock.ops:Updating [trial158.front.sepia.ceph.com]: reset os type and version on server
2026-02-12T08:41:18.830 INFO:teuthology.lock.ops:Updating trial158.front.sepia.ceph.com on lock server
2026-02-12T08:41:18.887 INFO:teuthology.lock.ops:Start node 'trial189.front.sepia.ceph.com' reimaging
2026-02-12T08:41:18.887 INFO:teuthology.lock.ops:Updating [trial189.front.sepia.ceph.com]: reset os type and version on server
2026-02-12T08:41:18.888 INFO:teuthology.lock.ops:Updating trial189.front.sepia.ceph.com on lock server
2026-02-12T08:41:19.120 DEBUG:teuthology.provision.fog.trial158:Requesting image trial_centos_9.stream (ID 3)
2026-02-12T08:41:19.129 DEBUG:teuthology.provision.fog.trial189:Requesting image trial_centos_9.stream (ID 3)
2026-02-12T08:41:19.169 INFO:teuthology.provision.fog.trial158:Scheduling deploy of centos 9.stream
2026-02-12T08:41:19.181 INFO:teuthology.provision.fog.trial189:Scheduling deploy of centos 9.stream
2026-02-12T08:41:20.095 INFO:teuthology.orchestra.console.trial158:Power off
2026-02-12T08:41:20.095 DEBUG:teuthology.orchestra.console.trial158:pexpect command: ipmitool -H trial158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T08:41:20.114 INFO:teuthology.orchestra.console.trial189:Power off
2026-02-12T08:41:20.114 DEBUG:teuthology.orchestra.console.trial189:pexpect command: ipmitool -H trial189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T08:41:20.138 DEBUG:teuthology.orchestra.console.trial158:power off output: Chassis Power Control: Down/Off
2026-02-12T08:41:20.152 DEBUG:teuthology.orchestra.console.trial189:power off output: Chassis Power Control: Down/Off
2026-02-12T08:41:24.143 DEBUG:teuthology.orchestra.console.trial158:pexpect command: ipmitool -H trial158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T08:41:24.153 DEBUG:teuthology.orchestra.console.trial189:pexpect command: ipmitool -H trial189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T08:41:24.252 DEBUG:teuthology.orchestra.console.trial158:check power output: Chassis Power is off
2026-02-12T08:41:24.253 INFO:teuthology.orchestra.console.trial158:Power off completed
2026-02-12T08:41:24.262 DEBUG:teuthology.orchestra.console.trial189:check power output: Chassis Power is off
2026-02-12T08:41:24.263 INFO:teuthology.orchestra.console.trial189:Power off completed
2026-02-12T08:41:24.354 INFO:teuthology.orchestra.console.trial158:Power on
2026-02-12T08:41:24.355 DEBUG:teuthology.orchestra.console.trial158:pexpect command: ipmitool -H trial158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-12T08:41:24.364 INFO:teuthology.orchestra.console.trial189:Power on
2026-02-12T08:41:24.364 DEBUG:teuthology.orchestra.console.trial189:pexpect command: ipmitool -H trial189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-12T08:41:24.395 DEBUG:teuthology.orchestra.console.trial158:power on output: Chassis Power Control: Up/On
2026-02-12T08:41:24.404 DEBUG:teuthology.orchestra.console.trial189:power on output: Chassis Power Control: Up/On
2026-02-12T08:41:28.400 DEBUG:teuthology.orchestra.console.trial158:pexpect command: ipmitool -H trial158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T08:41:28.409 DEBUG:teuthology.orchestra.console.trial189:pexpect command: ipmitool -H trial189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T08:41:28.512 DEBUG:teuthology.orchestra.console.trial158:check power output: Chassis Power is off
2026-02-12T08:41:28.517 DEBUG:teuthology.orchestra.console.trial189:check power output: Chassis Power is on
2026-02-12T08:41:28.518 INFO:teuthology.orchestra.console.trial189:Power on completed
2026-02-12T08:41:28.618 INFO:teuthology.provision.fog.trial189:Waiting for deploy to finish
2026-02-12T08:41:32.516 DEBUG:teuthology.orchestra.console.trial158:pexpect command: ipmitool -H trial158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T08:41:32.626 DEBUG:teuthology.orchestra.console.trial158:check power output: Chassis Power is on
2026-02-12T08:41:32.626 INFO:teuthology.orchestra.console.trial158:Power on completed
2026-02-12T08:41:32.727 INFO:teuthology.provision.fog.trial158:Waiting for deploy to finish
2026-02-12T08:43:15.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:43:18.493 WARNING:teuthology.provision.fog.trial189:[Errno None] Unable to connect to port 22 on 10.20.193.189
2026-02-12T08:43:19.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:43:22.525 WARNING:teuthology.provision.fog.trial158:[Errno None] Unable to connect to port 22 on 10.20.193.158
2026-02-12T08:43:24.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:43:27.581 WARNING:teuthology.provision.fog.trial189:[Errno None] Unable to connect to port 22 on 10.20.193.189
2026-02-12T08:43:28.526 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:43:31.613 WARNING:teuthology.provision.fog.trial158:[Errno None] Unable to connect to port 22 on 10.20.193.158
2026-02-12T08:43:33.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:43:36.669 WARNING:teuthology.provision.fog.trial189:[Errno None] Unable to connect to port 22 on 10.20.193.189
2026-02-12T08:43:37.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:43:40.702 WARNING:teuthology.provision.fog.trial158:[Errno None] Unable to connect to port 22 on 10.20.193.158
2026-02-12T08:43:42.672 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:43:45.757 WARNING:teuthology.provision.fog.trial189:[Errno None] Unable to connect to port 22 on 10.20.193.189
2026-02-12T08:43:46.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:43:49.789 WARNING:teuthology.provision.fog.trial158:[Errno None] Unable to connect to port 22 on 10.20.193.158
2026-02-12T08:43:51.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:43:55.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:44:27.796 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-12T08:44:27.797 DEBUG:teuthology.orchestra.run.trial189:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-12T08:44:31.901 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-12T08:44:31.901 DEBUG:teuthology.orchestra.run.trial158:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-12T08:44:32.062 INFO:teuthology.provision.fog.trial158:Node is ready
2026-02-12T08:44:32.062 DEBUG:teuthology.orchestra.run.trial158:> hostname
2026-02-12T08:44:32.117 INFO:teuthology.orchestra.run.trial158.stdout:trial158.front.sepia.ceph.com
2026-02-12T08:44:32.118 DEBUG:teuthology.orchestra.run.trial158:> grep trial158.front.sepia.ceph.com /etc/hosts
2026-02-12T08:44:32.174 INFO:teuthology.orchestra.run.trial158.stdout:10.20.193.158 trial158.front.sepia.ceph.com trial158
2026-02-12T08:44:32.175 DEBUG:teuthology.orchestra.run.trial158:> sudo hostname trial158
2026-02-12T08:44:32.244 DEBUG:teuthology.orchestra.run.trial158:> sudo sed -i -e 's/trial158.front.sepia.ceph.com/trial158/g' /etc/hosts
2026-02-12T08:44:32.308 DEBUG:teuthology.orchestra.run.trial158:> sudo sed -i -e 's/10.20.193.158/10.20.193.158/g' /etc/hosts
2026-02-12T08:44:32.373 DEBUG:teuthology.orchestra.run.trial158:> sudo sed -i -e 's/trial158.front.sepia.ceph.com/trial158/g' /etc/hostname
2026-02-12T08:44:32.437 DEBUG:teuthology.orchestra.run.trial158:> sudo hostname trial158
2026-02-12T08:44:32.499 DEBUG:teuthology.orchestra.run.trial158:> cat /etc/os-release
2026-02-12T08:44:32.555 INFO:teuthology.orchestra.run.trial158.stdout:NAME="CentOS Stream"
2026-02-12T08:44:32.555 INFO:teuthology.orchestra.run.trial158.stdout:VERSION="9"
2026-02-12T08:44:32.555 INFO:teuthology.orchestra.run.trial158.stdout:ID="centos"
2026-02-12T08:44:32.556 INFO:teuthology.orchestra.run.trial158.stdout:ID_LIKE="rhel fedora"
2026-02-12T08:44:32.556 INFO:teuthology.orchestra.run.trial158.stdout:VERSION_ID="9"
2026-02-12T08:44:32.556 INFO:teuthology.orchestra.run.trial158.stdout:PLATFORM_ID="platform:el9"
2026-02-12T08:44:32.556 INFO:teuthology.orchestra.run.trial158.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T08:44:32.556 INFO:teuthology.orchestra.run.trial158.stdout:ANSI_COLOR="0;31"
2026-02-12T08:44:32.556 INFO:teuthology.orchestra.run.trial158.stdout:LOGO="fedora-logo-icon"
2026-02-12T08:44:32.556 INFO:teuthology.orchestra.run.trial158.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T08:44:32.556 INFO:teuthology.orchestra.run.trial158.stdout:HOME_URL="https://centos.org/"
2026-02-12T08:44:32.556 INFO:teuthology.orchestra.run.trial158.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T08:44:32.556 INFO:teuthology.orchestra.run.trial158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T08:44:32.556 INFO:teuthology.orchestra.run.trial158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T08:44:32.556 INFO:teuthology.provision.fog.trial158:Deploy complete!
2026-02-12T08:44:32.557 DEBUG:teuthology.parallel:result is None
2026-02-12T08:44:32.964 INFO:teuthology.provision.fog.trial189:Node is ready
2026-02-12T08:44:32.965 DEBUG:teuthology.orchestra.run.trial189:> hostname
2026-02-12T08:44:32.983 INFO:teuthology.orchestra.run.trial189.stdout:trial189.front.sepia.ceph.com
2026-02-12T08:44:32.983 DEBUG:teuthology.orchestra.run.trial189:> grep trial189.front.sepia.ceph.com /etc/hosts
2026-02-12T08:44:33.039 INFO:teuthology.orchestra.run.trial189.stdout:10.20.193.189 trial189.front.sepia.ceph.com trial189
2026-02-12T08:44:33.040 DEBUG:teuthology.orchestra.run.trial189:> sudo hostname trial189
2026-02-12T08:44:33.110 DEBUG:teuthology.orchestra.run.trial189:> sudo sed -i -e 's/trial189.front.sepia.ceph.com/trial189/g' /etc/hosts
2026-02-12T08:44:33.176 DEBUG:teuthology.orchestra.run.trial189:> sudo sed -i -e 's/10.20.193.189/10.20.193.189/g' /etc/hosts
2026-02-12T08:44:33.241 DEBUG:teuthology.orchestra.run.trial189:> sudo sed -i -e 's/trial189.front.sepia.ceph.com/trial189/g' /etc/hostname
2026-02-12T08:44:33.304 DEBUG:teuthology.orchestra.run.trial189:> sudo hostname trial189
2026-02-12T08:44:33.367 DEBUG:teuthology.orchestra.run.trial189:> cat /etc/os-release
2026-02-12T08:44:33.423 INFO:teuthology.orchestra.run.trial189.stdout:NAME="CentOS Stream"
2026-02-12T08:44:33.423 INFO:teuthology.orchestra.run.trial189.stdout:VERSION="9"
2026-02-12T08:44:33.423 INFO:teuthology.orchestra.run.trial189.stdout:ID="centos"
2026-02-12T08:44:33.423 INFO:teuthology.orchestra.run.trial189.stdout:ID_LIKE="rhel fedora"
2026-02-12T08:44:33.423 INFO:teuthology.orchestra.run.trial189.stdout:VERSION_ID="9"
2026-02-12T08:44:33.423 INFO:teuthology.orchestra.run.trial189.stdout:PLATFORM_ID="platform:el9"
2026-02-12T08:44:33.424 INFO:teuthology.orchestra.run.trial189.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T08:44:33.424 INFO:teuthology.orchestra.run.trial189.stdout:ANSI_COLOR="0;31"
2026-02-12T08:44:33.424 INFO:teuthology.orchestra.run.trial189.stdout:LOGO="fedora-logo-icon"
2026-02-12T08:44:33.424 INFO:teuthology.orchestra.run.trial189.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T08:44:33.424 INFO:teuthology.orchestra.run.trial189.stdout:HOME_URL="https://centos.org/"
2026-02-12T08:44:33.424 INFO:teuthology.orchestra.run.trial189.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T08:44:33.424 INFO:teuthology.orchestra.run.trial189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T08:44:33.424 INFO:teuthology.orchestra.run.trial189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T08:44:33.425 INFO:teuthology.provision.fog.trial189:Deploy complete!
2026-02-12T08:44:33.426 DEBUG:teuthology.parallel:result is None
2026-02-12T08:44:33.813 INFO:teuthology.lock.ops:Checking trial158.front.sepia.ceph.com
2026-02-12T08:44:33.814 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-12T08:44:33.913 INFO:teuthology.lock.ops:Checking trial189.front.sepia.ceph.com
2026-02-12T08:44:33.913 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-12T08:44:33.976 INFO:teuthology.lock.ops:Updating [trial158.front.sepia.ceph.com]: set os type and version on server
2026-02-12T08:44:33.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial158.front.sepia.ceph.com'
2026-02-12T08:44:33.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:44:34.035 DEBUG:teuthology.orchestra.run.trial158:> true
2026-02-12T08:44:34.111 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial158.front.sepia.ceph.com'
2026-02-12T08:44:34.111 DEBUG:teuthology.orchestra.run.trial158:> uname -m
2026-02-12T08:44:34.166 INFO:teuthology.orchestra.run.trial158.stdout:x86_64
2026-02-12T08:44:34.167 DEBUG:teuthology.orchestra.run.trial158:> cat /etc/os-release
2026-02-12T08:44:34.223 INFO:teuthology.orchestra.run.trial158.stdout:NAME="CentOS Stream"
2026-02-12T08:44:34.223 INFO:teuthology.orchestra.run.trial158.stdout:VERSION="9"
2026-02-12T08:44:34.223 INFO:teuthology.orchestra.run.trial158.stdout:ID="centos"
2026-02-12T08:44:34.223 INFO:teuthology.orchestra.run.trial158.stdout:ID_LIKE="rhel fedora"
2026-02-12T08:44:34.223 INFO:teuthology.orchestra.run.trial158.stdout:VERSION_ID="9"
2026-02-12T08:44:34.223 INFO:teuthology.orchestra.run.trial158.stdout:PLATFORM_ID="platform:el9"
2026-02-12T08:44:34.223 INFO:teuthology.orchestra.run.trial158.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T08:44:34.224 INFO:teuthology.orchestra.run.trial158.stdout:ANSI_COLOR="0;31"
2026-02-12T08:44:34.224 INFO:teuthology.orchestra.run.trial158.stdout:LOGO="fedora-logo-icon"
2026-02-12T08:44:34.224 INFO:teuthology.orchestra.run.trial158.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T08:44:34.224 INFO:teuthology.orchestra.run.trial158.stdout:HOME_URL="https://centos.org/"
2026-02-12T08:44:34.224 INFO:teuthology.orchestra.run.trial158.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T08:44:34.224 INFO:teuthology.orchestra.run.trial158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T08:44:34.224 INFO:teuthology.orchestra.run.trial158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T08:44:34.225 INFO:teuthology.lock.ops:Updating trial158.front.sepia.ceph.com on lock server
2026-02-12T08:44:34.328 INFO:teuthology.lock.ops:Updating [trial189.front.sepia.ceph.com]: set os type and version on server
2026-02-12T08:44:34.328 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial189.front.sepia.ceph.com'
2026-02-12T08:44:34.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:44:34.397 DEBUG:teuthology.orchestra.run.trial189:> true
2026-02-12T08:44:34.470 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial189.front.sepia.ceph.com'
2026-02-12T08:44:34.471 DEBUG:teuthology.orchestra.run.trial189:> uname -m
2026-02-12T08:44:34.527 INFO:teuthology.orchestra.run.trial189.stdout:x86_64
2026-02-12T08:44:34.527 DEBUG:teuthology.orchestra.run.trial189:> cat /etc/os-release
2026-02-12T08:44:34.583 INFO:teuthology.orchestra.run.trial189.stdout:NAME="CentOS Stream"
2026-02-12T08:44:34.584 INFO:teuthology.orchestra.run.trial189.stdout:VERSION="9"
2026-02-12T08:44:34.584 INFO:teuthology.orchestra.run.trial189.stdout:ID="centos"
2026-02-12T08:44:34.584 INFO:teuthology.orchestra.run.trial189.stdout:ID_LIKE="rhel fedora"
2026-02-12T08:44:34.584 INFO:teuthology.orchestra.run.trial189.stdout:VERSION_ID="9"
2026-02-12T08:44:34.584 INFO:teuthology.orchestra.run.trial189.stdout:PLATFORM_ID="platform:el9"
2026-02-12T08:44:34.584 INFO:teuthology.orchestra.run.trial189.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T08:44:34.584 INFO:teuthology.orchestra.run.trial189.stdout:ANSI_COLOR="0;31"
2026-02-12T08:44:34.584 INFO:teuthology.orchestra.run.trial189.stdout:LOGO="fedora-logo-icon"
2026-02-12T08:44:34.584 INFO:teuthology.orchestra.run.trial189.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T08:44:34.585 INFO:teuthology.orchestra.run.trial189.stdout:HOME_URL="https://centos.org/"
2026-02-12T08:44:34.585 INFO:teuthology.orchestra.run.trial189.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T08:44:34.585 INFO:teuthology.orchestra.run.trial189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T08:44:34.585 INFO:teuthology.orchestra.run.trial189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T08:44:34.586 INFO:teuthology.lock.ops:Updating trial189.front.sepia.ceph.com on lock server
2026-02-12T08:44:34.689 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:44:34.785 INFO:teuthology.dispatcher.supervisor:Running job 41579
2026-02-12T08:44:34.785 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_c433f1062990a0488dc29a553589bc609a460691/virtualenv/bin/teuthology -v --owner scheduled_matan@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41579 --name matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial --description crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented} thrashers/{default} thrashosds-health workloads/snaps-few-objects-balanced} -- /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41579/orig.config.yaml
2026-02-12T08:44:34.785 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41579
2026-02-12T08:44:34.785 INFO:teuthology.dispatcher.supervisor:Job PID: 3370980
2026-02-12T08:44:34.785 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-12T08:46:34.886 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:48:35.087 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:50:35.291 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:52:35.408 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:54:35.612 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:56:35.731 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:58:35.935 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:00:36.033 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:02:36.217 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:04:36.352 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:06:36.532 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:08:36.633 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:10:36.837 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:12:37.025 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:14:37.206 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:16:37.384 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:18:37.583 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:20:37.779 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:22:37.906 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:24:38.076 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:26:38.191 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:28:38.358 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:30:38.493 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:32:38.597 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:34:38.771 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:36:38.993 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:38:39.204 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:40:39.403 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:42:39.537 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:44:39.760 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:46:39.975 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:48:40.151 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:50:40.271 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:52:40.403 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:54:40.580 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:56:40.789 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:58:40.908 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T10:00:41.109 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T10:02:41.335 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T10:04:41.611 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T10:04:41.705 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-12T10:04:41.820 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-12T10:04:42.322 INFO:teuthology.orchestra.console.trial189:Power off
2026-02-12T10:04:42.322 DEBUG:teuthology.orchestra.console.trial189:pexpect command: ipmitool -H trial189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T10:04:42.342 INFO:teuthology.orchestra.console.trial158:Power off
2026-02-12T10:04:42.342 DEBUG:teuthology.orchestra.console.trial158:pexpect command: ipmitool -H trial158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T10:04:42.361 DEBUG:teuthology.orchestra.console.trial189:power off output: Chassis Power Control: Down/Off
2026-02-12T10:04:42.380 DEBUG:teuthology.orchestra.console.trial158:power off output: Chassis Power Control: Down/Off
2026-02-12T10:04:46.367 DEBUG:teuthology.orchestra.console.trial189:pexpect command: ipmitool -H trial189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T10:04:46.380 DEBUG:teuthology.orchestra.console.trial158:pexpect command: ipmitool -H trial158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T10:04:46.475 DEBUG:teuthology.orchestra.console.trial189:check power output: Chassis Power is on
2026-02-12T10:04:46.489 DEBUG:teuthology.orchestra.console.trial158:check power output: Chassis Power is on
2026-02-12T10:04:50.480 DEBUG:teuthology.orchestra.console.trial189:pexpect command: ipmitool -H trial189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T10:04:50.490 DEBUG:teuthology.orchestra.console.trial158:pexpect command: ipmitool -H trial158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T10:04:50.590 DEBUG:teuthology.orchestra.console.trial189:check power output: Chassis Power is off
2026-02-12T10:04:50.590 INFO:teuthology.orchestra.console.trial189:Power off completed
2026-02-12T10:04:50.600 DEBUG:teuthology.orchestra.console.trial158:check power output: Chassis Power is off
2026-02-12T10:04:50.600 INFO:teuthology.orchestra.console.trial158:Power off completed
2026-02-12T10:04:50.836 INFO:teuthology.lock.ops:unlocked: trial189.front.sepia.ceph.com
2026-02-12T10:04:50.849 INFO:teuthology.lock.ops:unlocked: trial158.front.sepia.ceph.com