2026-02-12T09:21:03.505 INFO:root:teuthology version: 0.0.1.dev29+gc433f1062
2026-02-12T09:21:03.505 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:21:03.577 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:21:03.782 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x708329542840>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x708328d836a0>, signals=[15])
2026-02-12T09:21:03.782 INFO:teuthology.lock.ops:Start node 'trial007.front.sepia.ceph.com' reimaging
2026-02-12T09:21:03.782 INFO:teuthology.lock.ops:Updating [trial007.front.sepia.ceph.com]: reset os type and version on server
2026-02-12T09:21:03.782 INFO:teuthology.lock.ops:Updating trial007.front.sepia.ceph.com on lock server
2026-02-12T09:21:03.839 INFO:teuthology.lock.ops:Start node 'trial107.front.sepia.ceph.com' reimaging
2026-02-12T09:21:03.839 INFO:teuthology.lock.ops:Updating [trial107.front.sepia.ceph.com]: reset os type and version on server
2026-02-12T09:21:03.839 INFO:teuthology.lock.ops:Updating trial107.front.sepia.ceph.com on lock server
2026-02-12T09:21:04.064 DEBUG:teuthology.provision.fog.trial007:Requesting image trial_centos_9.stream (ID 3)
2026-02-12T09:21:04.079 DEBUG:teuthology.provision.fog.trial107:Requesting image trial_centos_9.stream (ID 3)
2026-02-12T09:21:04.097 INFO:teuthology.provision.fog.trial107:Scheduling deploy of centos 9.stream
2026-02-12T09:21:04.111 INFO:teuthology.provision.fog.trial007:Scheduling deploy of centos 9.stream
2026-02-12T09:21:05.090 INFO:teuthology.orchestra.console.trial007:Power off
2026-02-12T09:21:05.090 DEBUG:teuthology.orchestra.console.trial007:pexpect command: ipmitool -H trial007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T09:21:05.108 INFO:teuthology.orchestra.console.trial107:Power off
2026-02-12T09:21:05.108 DEBUG:teuthology.orchestra.console.trial107:pexpect command: ipmitool -H trial107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T09:21:05.127 DEBUG:teuthology.orchestra.console.trial007:power off output: Chassis Power Control: Down/Off
2026-02-12T09:21:05.146 DEBUG:teuthology.orchestra.console.trial107:power off output: Chassis Power Control: Down/Off
2026-02-12T09:21:09.133 DEBUG:teuthology.orchestra.console.trial007:pexpect command: ipmitool -H trial007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T09:21:09.147 DEBUG:teuthology.orchestra.console.trial107:pexpect command: ipmitool -H trial107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T09:21:09.242 DEBUG:teuthology.orchestra.console.trial007:check power output: Chassis Power is off
2026-02-12T09:21:09.242 INFO:teuthology.orchestra.console.trial007:Power off completed
2026-02-12T09:21:09.254 DEBUG:teuthology.orchestra.console.trial107:check power output: Chassis Power is off
2026-02-12T09:21:09.254 INFO:teuthology.orchestra.console.trial107:Power off completed
2026-02-12T09:21:09.344 INFO:teuthology.orchestra.console.trial007:Power on
2026-02-12T09:21:09.344 DEBUG:teuthology.orchestra.console.trial007:pexpect command: ipmitool -H trial007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-12T09:21:09.355 INFO:teuthology.orchestra.console.trial107:Power on
2026-02-12T09:21:09.356 DEBUG:teuthology.orchestra.console.trial107:pexpect command: ipmitool -H trial107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-12T09:21:09.384 DEBUG:teuthology.orchestra.console.trial007:power on output: Chassis Power Control: Up/On
2026-02-12T09:21:09.395 DEBUG:teuthology.orchestra.console.trial107:power on output: Chassis Power Control: Up/On
2026-02-12T09:21:13.389 DEBUG:teuthology.orchestra.console.trial007:pexpect command: ipmitool -H trial007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T09:21:13.398 DEBUG:teuthology.orchestra.console.trial107:pexpect command: ipmitool -H trial107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T09:21:13.500 DEBUG:teuthology.orchestra.console.trial007:check power output: Chassis Power is on
2026-02-12T09:21:13.501 INFO:teuthology.orchestra.console.trial007:Power on completed
2026-02-12T09:21:13.508 DEBUG:teuthology.orchestra.console.trial107:check power output: Chassis Power is on
2026-02-12T09:21:13.509 INFO:teuthology.orchestra.console.trial107:Power on completed
2026-02-12T09:21:13.602 INFO:teuthology.provision.fog.trial007:Waiting for deploy to finish
2026-02-12T09:21:13.609 INFO:teuthology.provision.fog.trial107:Waiting for deploy to finish
2026-02-12T09:23:00.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:23:00.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:23:03.965 WARNING:teuthology.provision.fog.trial007:[Errno None] Unable to connect to port 22 on 10.20.193.7
2026-02-12T09:23:03.965 WARNING:teuthology.provision.fog.trial107:[Errno None] Unable to connect to port 22 on 10.20.193.107
2026-02-12T09:23:09.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:23:09.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:23:13.053 WARNING:teuthology.provision.fog.trial007:[Errno None] Unable to connect to port 22 on 10.20.193.7
2026-02-12T09:23:13.053 WARNING:teuthology.provision.fog.trial107:[Errno None] Unable to connect to port 22 on 10.20.193.107
2026-02-12T09:23:19.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:23:19.060 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:23:22.141 WARNING:teuthology.provision.fog.trial007:[Errno None] Unable to connect to port 22 on 10.20.193.7
2026-02-12T09:23:22.141 WARNING:teuthology.provision.fog.trial107:[Errno None] Unable to connect to port 22 on 10.20.193.107
2026-02-12T09:23:28.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:23:28.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:23:31.229 WARNING:teuthology.provision.fog.trial007:[Errno None] Unable to connect to port 22 on 10.20.193.7
2026-02-12T09:23:31.229 WARNING:teuthology.provision.fog.trial107:[Errno None] Unable to connect to port 22 on 10.20.193.107
2026-02-12T09:23:37.236 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:23:37.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:24:13.258 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-12T09:24:13.258 DEBUG:teuthology.orchestra.run.trial107:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-12T09:24:13.263 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-12T09:24:13.263 DEBUG:teuthology.orchestra.run.trial007:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-12T09:24:13.484 INFO:teuthology.provision.fog.trial107:Node is ready
2026-02-12T09:24:13.484 DEBUG:teuthology.orchestra.run.trial107:> hostname
2026-02-12T09:24:13.497 INFO:teuthology.provision.fog.trial007:Node is ready
2026-02-12T09:24:13.497 DEBUG:teuthology.orchestra.run.trial007:> hostname
2026-02-12T09:24:13.540 INFO:teuthology.orchestra.run.trial107.stdout:trial107.front.sepia.ceph.com
2026-02-12T09:24:13.540 DEBUG:teuthology.orchestra.run.trial107:> grep trial107.front.sepia.ceph.com /etc/hosts
2026-02-12T09:24:13.554 INFO:teuthology.orchestra.run.trial007.stdout:trial007.front.sepia.ceph.com
2026-02-12T09:24:13.554 DEBUG:teuthology.orchestra.run.trial007:> grep trial007.front.sepia.ceph.com /etc/hosts
2026-02-12T09:24:13.597 INFO:teuthology.orchestra.run.trial107.stdout:10.20.193.107 trial107.front.sepia.ceph.com trial107
2026-02-12T09:24:13.598 DEBUG:teuthology.orchestra.run.trial107:> sudo hostname trial107
2026-02-12T09:24:13.609 INFO:teuthology.orchestra.run.trial007.stdout:10.20.193.7 trial007.front.sepia.ceph.com trial007
2026-02-12T09:24:13.610 DEBUG:teuthology.orchestra.run.trial007:> sudo hostname trial007
2026-02-12T09:24:13.681 DEBUG:teuthology.orchestra.run.trial007:> sudo sed -i -e 's/trial007.front.sepia.ceph.com/trial007/g' /etc/hosts
2026-02-12T09:24:13.724 DEBUG:teuthology.orchestra.run.trial107:> sudo sed -i -e 's/trial107.front.sepia.ceph.com/trial107/g' /etc/hosts
2026-02-12T09:24:13.747 DEBUG:teuthology.orchestra.run.trial007:> sudo sed -i -e 's/10.20.193.7/10.20.193.7/g' /etc/hosts
2026-02-12T09:24:13.784 DEBUG:teuthology.orchestra.run.trial107:> sudo sed -i -e 's/10.20.193.107/10.20.193.107/g' /etc/hosts
2026-02-12T09:24:13.813 DEBUG:teuthology.orchestra.run.trial007:> sudo sed -i -e 's/trial007.front.sepia.ceph.com/trial007/g' /etc/hostname
2026-02-12T09:24:13.818 DEBUG:teuthology.orchestra.run.trial107:> sudo sed -i -e 's/trial107.front.sepia.ceph.com/trial107/g' /etc/hostname
2026-02-12T09:24:13.878 DEBUG:teuthology.orchestra.run.trial007:> sudo hostname trial007
2026-02-12T09:24:13.884 DEBUG:teuthology.orchestra.run.trial107:> sudo hostname trial107
2026-02-12T09:24:13.943 DEBUG:teuthology.orchestra.run.trial007:> cat /etc/os-release
2026-02-12T09:24:13.950 DEBUG:teuthology.orchestra.run.trial107:> cat /etc/os-release
2026-02-12T09:24:13.999 INFO:teuthology.orchestra.run.trial007.stdout:NAME="CentOS Stream"
2026-02-12T09:24:13.999 INFO:teuthology.orchestra.run.trial007.stdout:VERSION="9"
2026-02-12T09:24:13.999 INFO:teuthology.orchestra.run.trial007.stdout:ID="centos"
2026-02-12T09:24:13.999 INFO:teuthology.orchestra.run.trial007.stdout:ID_LIKE="rhel fedora"
2026-02-12T09:24:14.000 INFO:teuthology.orchestra.run.trial007.stdout:VERSION_ID="9"
2026-02-12T09:24:14.000 INFO:teuthology.orchestra.run.trial007.stdout:PLATFORM_ID="platform:el9"
2026-02-12T09:24:14.000 INFO:teuthology.orchestra.run.trial007.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T09:24:14.000 INFO:teuthology.orchestra.run.trial007.stdout:ANSI_COLOR="0;31"
2026-02-12T09:24:14.000 INFO:teuthology.orchestra.run.trial007.stdout:LOGO="fedora-logo-icon"
2026-02-12T09:24:14.000 INFO:teuthology.orchestra.run.trial007.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T09:24:14.000 INFO:teuthology.orchestra.run.trial007.stdout:HOME_URL="https://centos.org/"
2026-02-12T09:24:14.000 INFO:teuthology.orchestra.run.trial007.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T09:24:14.000 INFO:teuthology.orchestra.run.trial007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T09:24:14.000 INFO:teuthology.orchestra.run.trial007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T09:24:14.001 INFO:teuthology.provision.fog.trial007:Deploy complete!
2026-02-12T09:24:14.003 DEBUG:teuthology.parallel:result is None
2026-02-12T09:24:14.007 INFO:teuthology.orchestra.run.trial107.stdout:NAME="CentOS Stream"
2026-02-12T09:24:14.007 INFO:teuthology.orchestra.run.trial107.stdout:VERSION="9"
2026-02-12T09:24:14.007 INFO:teuthology.orchestra.run.trial107.stdout:ID="centos"
2026-02-12T09:24:14.007 INFO:teuthology.orchestra.run.trial107.stdout:ID_LIKE="rhel fedora"
2026-02-12T09:24:14.007 INFO:teuthology.orchestra.run.trial107.stdout:VERSION_ID="9"
2026-02-12T09:24:14.007 INFO:teuthology.orchestra.run.trial107.stdout:PLATFORM_ID="platform:el9"
2026-02-12T09:24:14.007 INFO:teuthology.orchestra.run.trial107.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T09:24:14.008 INFO:teuthology.orchestra.run.trial107.stdout:ANSI_COLOR="0;31"
2026-02-12T09:24:14.008 INFO:teuthology.orchestra.run.trial107.stdout:LOGO="fedora-logo-icon"
2026-02-12T09:24:14.008 INFO:teuthology.orchestra.run.trial107.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T09:24:14.008 INFO:teuthology.orchestra.run.trial107.stdout:HOME_URL="https://centos.org/"
2026-02-12T09:24:14.008 INFO:teuthology.orchestra.run.trial107.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T09:24:14.008 INFO:teuthology.orchestra.run.trial107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T09:24:14.008 INFO:teuthology.orchestra.run.trial107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T09:24:14.009 INFO:teuthology.provision.fog.trial107:Deploy complete!
2026-02-12T09:24:14.009 DEBUG:teuthology.parallel:result is None
2026-02-12T09:24:14.417 INFO:teuthology.lock.ops:Checking trial007.front.sepia.ceph.com
2026-02-12T09:24:14.417 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-12T09:24:14.514 INFO:teuthology.lock.ops:Checking trial107.front.sepia.ceph.com
2026-02-12T09:24:14.514 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-12T09:24:14.576 INFO:teuthology.lock.ops:Updating [trial007.front.sepia.ceph.com]: set os type and version on server
2026-02-12T09:24:14.576 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial007.front.sepia.ceph.com'
2026-02-12T09:24:14.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:24:14.641 DEBUG:teuthology.orchestra.run.trial007:> true
2026-02-12T09:24:14.720 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial007.front.sepia.ceph.com'
2026-02-12T09:24:14.720 DEBUG:teuthology.orchestra.run.trial007:> uname -m
2026-02-12T09:24:14.776 INFO:teuthology.orchestra.run.trial007.stdout:x86_64
2026-02-12T09:24:14.777 DEBUG:teuthology.orchestra.run.trial007:> cat /etc/os-release
2026-02-12T09:24:14.833 INFO:teuthology.orchestra.run.trial007.stdout:NAME="CentOS Stream"
2026-02-12T09:24:14.833 INFO:teuthology.orchestra.run.trial007.stdout:VERSION="9"
2026-02-12T09:24:14.833 INFO:teuthology.orchestra.run.trial007.stdout:ID="centos"
2026-02-12T09:24:14.833 INFO:teuthology.orchestra.run.trial007.stdout:ID_LIKE="rhel fedora"
2026-02-12T09:24:14.833 INFO:teuthology.orchestra.run.trial007.stdout:VERSION_ID="9"
2026-02-12T09:24:14.834 INFO:teuthology.orchestra.run.trial007.stdout:PLATFORM_ID="platform:el9"
2026-02-12T09:24:14.834 INFO:teuthology.orchestra.run.trial007.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T09:24:14.834 INFO:teuthology.orchestra.run.trial007.stdout:ANSI_COLOR="0;31"
2026-02-12T09:24:14.834 INFO:teuthology.orchestra.run.trial007.stdout:LOGO="fedora-logo-icon"
2026-02-12T09:24:14.834 INFO:teuthology.orchestra.run.trial007.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T09:24:14.834 INFO:teuthology.orchestra.run.trial007.stdout:HOME_URL="https://centos.org/"
2026-02-12T09:24:14.834 INFO:teuthology.orchestra.run.trial007.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T09:24:14.834 INFO:teuthology.orchestra.run.trial007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T09:24:14.834 INFO:teuthology.orchestra.run.trial007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T09:24:14.835 INFO:teuthology.lock.ops:Updating trial007.front.sepia.ceph.com on lock server
2026-02-12T09:24:14.937 INFO:teuthology.lock.ops:Updating [trial107.front.sepia.ceph.com]: set os type and version on server
2026-02-12T09:24:14.938 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial107.front.sepia.ceph.com'
2026-02-12T09:24:14.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:24:15.009 DEBUG:teuthology.orchestra.run.trial107:> true
2026-02-12T09:24:15.085 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial107.front.sepia.ceph.com'
2026-02-12T09:24:15.085 DEBUG:teuthology.orchestra.run.trial107:> uname -m
2026-02-12T09:24:15.140 INFO:teuthology.orchestra.run.trial107.stdout:x86_64
2026-02-12T09:24:15.140 DEBUG:teuthology.orchestra.run.trial107:> cat /etc/os-release
2026-02-12T09:24:15.197 INFO:teuthology.orchestra.run.trial107.stdout:NAME="CentOS Stream"
2026-02-12T09:24:15.197 INFO:teuthology.orchestra.run.trial107.stdout:VERSION="9"
2026-02-12T09:24:15.197 INFO:teuthology.orchestra.run.trial107.stdout:ID="centos"
2026-02-12T09:24:15.197 INFO:teuthology.orchestra.run.trial107.stdout:ID_LIKE="rhel fedora"
2026-02-12T09:24:15.197 INFO:teuthology.orchestra.run.trial107.stdout:VERSION_ID="9"
2026-02-12T09:24:15.197 INFO:teuthology.orchestra.run.trial107.stdout:PLATFORM_ID="platform:el9"
2026-02-12T09:24:15.197 INFO:teuthology.orchestra.run.trial107.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T09:24:15.197 INFO:teuthology.orchestra.run.trial107.stdout:ANSI_COLOR="0;31"
2026-02-12T09:24:15.197 INFO:teuthology.orchestra.run.trial107.stdout:LOGO="fedora-logo-icon"
2026-02-12T09:24:15.198 INFO:teuthology.orchestra.run.trial107.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T09:24:15.198 INFO:teuthology.orchestra.run.trial107.stdout:HOME_URL="https://centos.org/"
2026-02-12T09:24:15.198 INFO:teuthology.orchestra.run.trial107.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T09:24:15.198 INFO:teuthology.orchestra.run.trial107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T09:24:15.198 INFO:teuthology.orchestra.run.trial107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T09:24:15.199 INFO:teuthology.lock.ops:Updating trial107.front.sepia.ceph.com on lock server
2026-02-12T09:24:15.302 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:24:15.397 INFO:teuthology.dispatcher.supervisor:Running job 41654
2026-02-12T09:24:15.397 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_c433f1062990a0488dc29a553589bc609a460691/virtualenv/bin/teuthology -v --owner scheduled_matan@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41654 --name matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial --description crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/crimson_bluestore thrashers/{pggrow} thrashosds-health workloads/snaps-few-objects-localized} -- /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41654/orig.config.yaml
2026-02-12T09:24:15.398 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41654
2026-02-12T09:24:15.398 INFO:teuthology.dispatcher.supervisor:Job PID: 3460762
2026-02-12T09:24:15.398 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-12T09:26:15.493 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:28:15.640 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:30:15.803 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:32:15.901 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:34:16.097 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:36:16.311 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:38:16.427 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:40:16.568 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:42:16.772 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:44:16.925 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:46:17.075 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:48:17.234 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:50:17.412 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:52:17.595 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:54:17.777 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:54:17.868 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-12T09:54:17.986 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-12T09:54:18.495 INFO:teuthology.orchestra.console.trial007:Power off
2026-02-12T09:54:18.495 DEBUG:teuthology.orchestra.console.trial007:pexpect command: ipmitool -H trial007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T09:54:18.512 INFO:teuthology.orchestra.console.trial107:Power off
2026-02-12T09:54:18.512 DEBUG:teuthology.orchestra.console.trial107:pexpect command: ipmitool -H trial107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T09:54:18.532 DEBUG:teuthology.orchestra.console.trial007:power off output: Chassis Power Control: Down/Off
2026-02-12T09:54:18.551 DEBUG:teuthology.orchestra.console.trial107:power off output: Chassis Power Control: Down/Off
2026-02-12T09:54:22.538 DEBUG:teuthology.orchestra.console.trial007:pexpect command: ipmitool -H trial007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T09:54:22.551 DEBUG:teuthology.orchestra.console.trial107:pexpect command: ipmitool -H trial107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T09:54:22.646 DEBUG:teuthology.orchestra.console.trial007:check power output: Chassis Power is on
2026-02-12T09:54:22.659 DEBUG:teuthology.orchestra.console.trial107:check power output: Chassis Power is on
2026-02-12T09:54:26.651 DEBUG:teuthology.orchestra.console.trial007:pexpect command: ipmitool -H trial007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T09:54:26.660 DEBUG:teuthology.orchestra.console.trial107:pexpect command: ipmitool -H trial107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T09:54:26.759 DEBUG:teuthology.orchestra.console.trial007:check power output: Chassis Power is off
2026-02-12T09:54:26.759 INFO:teuthology.orchestra.console.trial007:Power off completed
2026-02-12T09:54:26.766 DEBUG:teuthology.orchestra.console.trial107:check power output: Chassis Power is off
2026-02-12T09:54:26.766 INFO:teuthology.orchestra.console.trial107:Power off completed
2026-02-12T09:54:26.956 INFO:teuthology.lock.ops:unlocked: trial007.front.sepia.ceph.com
2026-02-12T09:54:26.969 INFO:teuthology.lock.ops:unlocked: trial107.front.sepia.ceph.com