2026-02-12T08:37:11.076 INFO:root:teuthology version: 0.0.1.dev29+gc433f1062
2026-02-12T08:37:11.077 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:37:11.160 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:37:11.352 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7dc679d10290>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7dc679c6b9c0>, signals=[15])
2026-02-12T08:37:11.353 INFO:teuthology.lock.ops:Start node 'trial014.front.sepia.ceph.com' reimaging
2026-02-12T08:37:11.353 INFO:teuthology.lock.ops:Updating [trial014.front.sepia.ceph.com]: reset os type and version on server
2026-02-12T08:37:11.353 INFO:teuthology.lock.ops:Updating trial014.front.sepia.ceph.com on lock server
2026-02-12T08:37:11.409 INFO:teuthology.lock.ops:Start node 'trial163.front.sepia.ceph.com' reimaging
2026-02-12T08:37:11.409 INFO:teuthology.lock.ops:Updating [trial163.front.sepia.ceph.com]: reset os type and version on server
2026-02-12T08:37:11.409 INFO:teuthology.lock.ops:Updating trial163.front.sepia.ceph.com on lock server
2026-02-12T08:37:11.640 DEBUG:teuthology.provision.fog.trial014:Requesting image trial_centos_9.stream (ID 3)
2026-02-12T08:37:11.648 DEBUG:teuthology.provision.fog.trial163:Requesting image trial_centos_9.stream (ID 3)
2026-02-12T08:37:11.692 INFO:teuthology.provision.fog.trial014:Scheduling deploy of centos 9.stream
2026-02-12T08:37:11.700 INFO:teuthology.provision.fog.trial163:Scheduling deploy of centos 9.stream
2026-02-12T08:37:12.769 INFO:teuthology.orchestra.console.trial014:Power off
2026-02-12T08:37:12.769 DEBUG:teuthology.orchestra.console.trial014:pexpect command: ipmitool -H trial014.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T08:37:12.791 INFO:teuthology.orchestra.console.trial163:Power off
2026-02-12T08:37:12.791 DEBUG:teuthology.orchestra.console.trial163:pexpect command: ipmitool -H trial163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T08:37:12.808 DEBUG:teuthology.orchestra.console.trial014:power off output: Chassis Power Control: Down/Off
2026-02-12T08:37:12.828 DEBUG:teuthology.orchestra.console.trial163:power off output: Chassis Power Control: Down/Off
2026-02-12T08:37:16.812 DEBUG:teuthology.orchestra.console.trial014:pexpect command: ipmitool -H trial014.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T08:37:16.828 DEBUG:teuthology.orchestra.console.trial163:pexpect command: ipmitool -H trial163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T08:37:16.922 DEBUG:teuthology.orchestra.console.trial014:check power output: Chassis Power is off
2026-02-12T08:37:16.922 INFO:teuthology.orchestra.console.trial014:Power off completed
2026-02-12T08:37:16.936 DEBUG:teuthology.orchestra.console.trial163:check power output: Chassis Power is off
2026-02-12T08:37:16.936 INFO:teuthology.orchestra.console.trial163:Power off completed
2026-02-12T08:37:17.023 INFO:teuthology.orchestra.console.trial014:Power on
2026-02-12T08:37:17.023 DEBUG:teuthology.orchestra.console.trial014:pexpect command: ipmitool -H trial014.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-12T08:37:17.037 INFO:teuthology.orchestra.console.trial163:Power on
2026-02-12T08:37:17.037 DEBUG:teuthology.orchestra.console.trial163:pexpect command: ipmitool -H trial163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-12T08:37:17.063 DEBUG:teuthology.orchestra.console.trial014:power on output: Chassis Power Control: Up/On
2026-02-12T08:37:17.083 DEBUG:teuthology.orchestra.console.trial163:power on output: Chassis Power Control: Up/On
2026-02-12T08:37:21.069 DEBUG:teuthology.orchestra.console.trial014:pexpect command: ipmitool -H trial014.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T08:37:21.084 DEBUG:teuthology.orchestra.console.trial163:pexpect command: ipmitool -H trial163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T08:37:21.179 DEBUG:teuthology.orchestra.console.trial014:check power output: Chassis Power is on
2026-02-12T08:37:21.179 INFO:teuthology.orchestra.console.trial014:Power on completed
2026-02-12T08:37:21.192 DEBUG:teuthology.orchestra.console.trial163:check power output: Chassis Power is on
2026-02-12T08:37:21.192 INFO:teuthology.orchestra.console.trial163:Power on completed
2026-02-12T08:37:21.281 INFO:teuthology.provision.fog.trial014:Waiting for deploy to finish
2026-02-12T08:37:21.293 INFO:teuthology.provision.fog.trial163:Waiting for deploy to finish
2026-02-12T08:39:08.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:39:08.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial014.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:39:11.645 WARNING:teuthology.provision.fog.trial163:[Errno None] Unable to connect to port 22 on 10.20.193.163
2026-02-12T08:39:11.773 WARNING:teuthology.provision.fog.trial014:[Errno None] Unable to connect to port 22 on 10.20.193.14
2026-02-12T08:39:17.652 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:39:17.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial014.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:39:20.734 WARNING:teuthology.provision.fog.trial163:[Errno None] Unable to connect to port 22 on 10.20.193.163
2026-02-12T08:39:20.861 WARNING:teuthology.provision.fog.trial014:[Errno None] Unable to connect to port 22 on 10.20.193.14
2026-02-12T08:39:26.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:39:26.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial014.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:39:29.821 WARNING:teuthology.provision.fog.trial163:[Errno None] Unable to connect to port 22 on 10.20.193.163
2026-02-12T08:39:29.949 WARNING:teuthology.provision.fog.trial014:[Errno None] Unable to connect to port 22 on 10.20.193.14
2026-02-12T08:39:35.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:39:35.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial014.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:39:38.909 WARNING:teuthology.provision.fog.trial163:[Errno None] Unable to connect to port 22 on 10.20.193.163
2026-02-12T08:39:39.037 WARNING:teuthology.provision.fog.trial014:[Errno None] Unable to connect to port 22 on 10.20.193.14
2026-02-12T08:39:44.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:39:45.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial014.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:40:21.052 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-12T08:40:21.053 DEBUG:teuthology.orchestra.run.trial014:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-12T08:40:21.072 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-12T08:40:21.072 DEBUG:teuthology.orchestra.run.trial163:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-12T08:40:21.263 INFO:teuthology.provision.fog.trial014:Node is ready
2026-02-12T08:40:21.264 DEBUG:teuthology.orchestra.run.trial014:> hostname
2026-02-12T08:40:21.287 INFO:teuthology.provision.fog.trial163:Node is ready
2026-02-12T08:40:21.287 DEBUG:teuthology.orchestra.run.trial163:> hostname
2026-02-12T08:40:21.318 INFO:teuthology.orchestra.run.trial014.stdout:trial014.front.sepia.ceph.com
2026-02-12T08:40:21.318 DEBUG:teuthology.orchestra.run.trial014:> grep trial014.front.sepia.ceph.com /etc/hosts
2026-02-12T08:40:21.346 INFO:teuthology.orchestra.run.trial163.stdout:trial163.front.sepia.ceph.com
2026-02-12T08:40:21.346 DEBUG:teuthology.orchestra.run.trial163:> grep trial163.front.sepia.ceph.com /etc/hosts
2026-02-12T08:40:21.374 INFO:teuthology.orchestra.run.trial014.stdout:10.20.193.14 trial014.front.sepia.ceph.com trial014
2026-02-12T08:40:21.374 DEBUG:teuthology.orchestra.run.trial014:> sudo hostname trial014
2026-02-12T08:40:21.405 INFO:teuthology.orchestra.run.trial163.stdout:10.20.193.163 trial163.front.sepia.ceph.com trial163
2026-02-12T08:40:21.405 DEBUG:teuthology.orchestra.run.trial163:> sudo hostname trial163
2026-02-12T08:40:21.443 DEBUG:teuthology.orchestra.run.trial014:> sudo sed -i -e 's/trial014.front.sepia.ceph.com/trial014/g' /etc/hosts
2026-02-12T08:40:21.478 DEBUG:teuthology.orchestra.run.trial163:> sudo sed -i -e 's/trial163.front.sepia.ceph.com/trial163/g' /etc/hosts
2026-02-12T08:40:21.510 DEBUG:teuthology.orchestra.run.trial014:> sudo sed -i -e 's/10.20.193.14/10.20.193.14/g' /etc/hosts
2026-02-12T08:40:21.546 DEBUG:teuthology.orchestra.run.trial163:> sudo sed -i -e 's/10.20.193.163/10.20.193.163/g' /etc/hosts
2026-02-12T08:40:21.574 DEBUG:teuthology.orchestra.run.trial014:> sudo sed -i -e 's/trial014.front.sepia.ceph.com/trial014/g' /etc/hostname
2026-02-12T08:40:21.612 DEBUG:teuthology.orchestra.run.trial163:> sudo sed -i -e 's/trial163.front.sepia.ceph.com/trial163/g' /etc/hostname
2026-02-12T08:40:21.639 DEBUG:teuthology.orchestra.run.trial014:> sudo hostname trial014
2026-02-12T08:40:21.679 DEBUG:teuthology.orchestra.run.trial163:> sudo hostname trial163
2026-02-12T08:40:21.702 DEBUG:teuthology.orchestra.run.trial014:> cat /etc/os-release
2026-02-12T08:40:21.746 DEBUG:teuthology.orchestra.run.trial163:> cat /etc/os-release
2026-02-12T08:40:21.759 INFO:teuthology.orchestra.run.trial014.stdout:NAME="CentOS Stream"
2026-02-12T08:40:21.759 INFO:teuthology.orchestra.run.trial014.stdout:VERSION="9"
2026-02-12T08:40:21.759 INFO:teuthology.orchestra.run.trial014.stdout:ID="centos"
2026-02-12T08:40:21.759 INFO:teuthology.orchestra.run.trial014.stdout:ID_LIKE="rhel fedora"
2026-02-12T08:40:21.759 INFO:teuthology.orchestra.run.trial014.stdout:VERSION_ID="9"
2026-02-12T08:40:21.759 INFO:teuthology.orchestra.run.trial014.stdout:PLATFORM_ID="platform:el9"
2026-02-12T08:40:21.759 INFO:teuthology.orchestra.run.trial014.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T08:40:21.759 INFO:teuthology.orchestra.run.trial014.stdout:ANSI_COLOR="0;31"
2026-02-12T08:40:21.759 INFO:teuthology.orchestra.run.trial014.stdout:LOGO="fedora-logo-icon"
2026-02-12T08:40:21.759 INFO:teuthology.orchestra.run.trial014.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T08:40:21.759 INFO:teuthology.orchestra.run.trial014.stdout:HOME_URL="https://centos.org/"
2026-02-12T08:40:21.759 INFO:teuthology.orchestra.run.trial014.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T08:40:21.760 INFO:teuthology.orchestra.run.trial014.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T08:40:21.760 INFO:teuthology.orchestra.run.trial014.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T08:40:21.760 INFO:teuthology.provision.fog.trial014:Deploy complete!
2026-02-12T08:40:21.760 DEBUG:teuthology.parallel:result is None
2026-02-12T08:40:21.805 INFO:teuthology.orchestra.run.trial163.stdout:NAME="CentOS Stream"
2026-02-12T08:40:21.805 INFO:teuthology.orchestra.run.trial163.stdout:VERSION="9"
2026-02-12T08:40:21.805 INFO:teuthology.orchestra.run.trial163.stdout:ID="centos"
2026-02-12T08:40:21.805 INFO:teuthology.orchestra.run.trial163.stdout:ID_LIKE="rhel fedora"
2026-02-12T08:40:21.805 INFO:teuthology.orchestra.run.trial163.stdout:VERSION_ID="9"
2026-02-12T08:40:21.805 INFO:teuthology.orchestra.run.trial163.stdout:PLATFORM_ID="platform:el9"
2026-02-12T08:40:21.805 INFO:teuthology.orchestra.run.trial163.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T08:40:21.805 INFO:teuthology.orchestra.run.trial163.stdout:ANSI_COLOR="0;31"
2026-02-12T08:40:21.805 INFO:teuthology.orchestra.run.trial163.stdout:LOGO="fedora-logo-icon"
2026-02-12T08:40:21.805 INFO:teuthology.orchestra.run.trial163.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T08:40:21.805 INFO:teuthology.orchestra.run.trial163.stdout:HOME_URL="https://centos.org/"
2026-02-12T08:40:21.805 INFO:teuthology.orchestra.run.trial163.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T08:40:21.805 INFO:teuthology.orchestra.run.trial163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T08:40:21.805 INFO:teuthology.orchestra.run.trial163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T08:40:21.805 INFO:teuthology.provision.fog.trial163:Deploy complete!
2026-02-12T08:40:21.806 DEBUG:teuthology.parallel:result is None
2026-02-12T08:40:22.158 INFO:teuthology.lock.ops:Checking trial014.front.sepia.ceph.com
2026-02-12T08:40:22.159 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-12T08:40:22.242 INFO:teuthology.lock.ops:Checking trial163.front.sepia.ceph.com
2026-02-12T08:40:22.243 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-12T08:40:22.300 INFO:teuthology.lock.ops:Updating [trial014.front.sepia.ceph.com]: set os type and version on server
2026-02-12T08:40:22.300 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial014.front.sepia.ceph.com'
2026-02-12T08:40:22.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial014.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:40:22.364 DEBUG:teuthology.orchestra.run.trial014:> true
2026-02-12T08:40:22.453 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial014.front.sepia.ceph.com'
2026-02-12T08:40:22.453 DEBUG:teuthology.orchestra.run.trial014:> uname -m
2026-02-12T08:40:22.509 INFO:teuthology.orchestra.run.trial014.stdout:x86_64
2026-02-12T08:40:22.509 DEBUG:teuthology.orchestra.run.trial014:> cat /etc/os-release
2026-02-12T08:40:22.606 INFO:teuthology.orchestra.run.trial014.stdout:NAME="CentOS Stream"
2026-02-12T08:40:22.607 INFO:teuthology.orchestra.run.trial014.stdout:VERSION="9"
2026-02-12T08:40:22.607 INFO:teuthology.orchestra.run.trial014.stdout:ID="centos"
2026-02-12T08:40:22.607 INFO:teuthology.orchestra.run.trial014.stdout:ID_LIKE="rhel fedora"
2026-02-12T08:40:22.607 INFO:teuthology.orchestra.run.trial014.stdout:VERSION_ID="9"
2026-02-12T08:40:22.607 INFO:teuthology.orchestra.run.trial014.stdout:PLATFORM_ID="platform:el9"
2026-02-12T08:40:22.607 INFO:teuthology.orchestra.run.trial014.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T08:40:22.607 INFO:teuthology.orchestra.run.trial014.stdout:ANSI_COLOR="0;31"
2026-02-12T08:40:22.607 INFO:teuthology.orchestra.run.trial014.stdout:LOGO="fedora-logo-icon"
2026-02-12T08:40:22.607 INFO:teuthology.orchestra.run.trial014.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T08:40:22.607 INFO:teuthology.orchestra.run.trial014.stdout:HOME_URL="https://centos.org/"
2026-02-12T08:40:22.608 INFO:teuthology.orchestra.run.trial014.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T08:40:22.608 INFO:teuthology.orchestra.run.trial014.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T08:40:22.608 INFO:teuthology.orchestra.run.trial014.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T08:40:22.608 INFO:teuthology.lock.ops:Updating trial014.front.sepia.ceph.com on lock server
2026-02-12T08:40:22.714 INFO:teuthology.lock.ops:Updating [trial163.front.sepia.ceph.com]: set os type and version on server
2026-02-12T08:40:22.714 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial163.front.sepia.ceph.com'
2026-02-12T08:40:22.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:40:22.784 DEBUG:teuthology.orchestra.run.trial163:> true
2026-02-12T08:40:22.869 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial163.front.sepia.ceph.com'
2026-02-12T08:40:22.869 DEBUG:teuthology.orchestra.run.trial163:> uname -m
2026-02-12T08:40:22.930 INFO:teuthology.orchestra.run.trial163.stdout:x86_64
2026-02-12T08:40:22.930 DEBUG:teuthology.orchestra.run.trial163:> cat /etc/os-release
2026-02-12T08:40:22.990 INFO:teuthology.orchestra.run.trial163.stdout:NAME="CentOS Stream"
2026-02-12T08:40:22.991 INFO:teuthology.orchestra.run.trial163.stdout:VERSION="9"
2026-02-12T08:40:22.991 INFO:teuthology.orchestra.run.trial163.stdout:ID="centos"
2026-02-12T08:40:22.991 INFO:teuthology.orchestra.run.trial163.stdout:ID_LIKE="rhel fedora"
2026-02-12T08:40:22.991 INFO:teuthology.orchestra.run.trial163.stdout:VERSION_ID="9"
2026-02-12T08:40:22.991 INFO:teuthology.orchestra.run.trial163.stdout:PLATFORM_ID="platform:el9"
2026-02-12T08:40:22.991 INFO:teuthology.orchestra.run.trial163.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T08:40:22.991 INFO:teuthology.orchestra.run.trial163.stdout:ANSI_COLOR="0;31"
2026-02-12T08:40:22.991 INFO:teuthology.orchestra.run.trial163.stdout:LOGO="fedora-logo-icon"
2026-02-12T08:40:22.991 INFO:teuthology.orchestra.run.trial163.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T08:40:22.991 INFO:teuthology.orchestra.run.trial163.stdout:HOME_URL="https://centos.org/"
2026-02-12T08:40:22.992 INFO:teuthology.orchestra.run.trial163.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T08:40:22.992 INFO:teuthology.orchestra.run.trial163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T08:40:22.992 INFO:teuthology.orchestra.run.trial163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T08:40:22.992 INFO:teuthology.lock.ops:Updating trial163.front.sepia.ceph.com on lock server
2026-02-12T08:40:23.095 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:40:23.190 INFO:teuthology.dispatcher.supervisor:Running job 41567
2026-02-12T08:40:23.190 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_c433f1062990a0488dc29a553589bc609a460691/virtualenv/bin/teuthology -v --owner scheduled_matan@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41567 --name matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial --description crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} thrashers/{default} thrashosds-health workloads/pool-snaps-few-objects} -- /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41567/orig.config.yaml
2026-02-12T08:40:23.190 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41567
2026-02-12T08:40:23.190 INFO:teuthology.dispatcher.supervisor:Job PID: 3353353
2026-02-12T08:40:23.190 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-12T08:42:23.277 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:44:23.443 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:46:23.589 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:48:23.797 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:50:24.004 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:52:24.133 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:54:24.325 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:56:24.498 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:58:24.670 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:00:24.868 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:02:25.022 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:04:25.223 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:06:25.330 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:08:25.518 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:10:25.714 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:12:25.885 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:14:26.071 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:16:26.256 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:18:26.418 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:20:26.595 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:22:26.778 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:24:26.869 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:26:27.037 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:28:27.230 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:30:27.316 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:32:27.457 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:34:27.577 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:36:27.751 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:38:27.864 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:40:28.066 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:42:28.157 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:44:28.328 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:46:28.534 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:48:28.713 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:50:28.898 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:50:28.979 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-12T09:50:29.100 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-12T09:50:29.636 INFO:teuthology.orchestra.console.trial014:Power off
2026-02-12T09:50:29.636 DEBUG:teuthology.orchestra.console.trial014:pexpect command: ipmitool -H trial014.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T09:50:29.657 INFO:teuthology.orchestra.console.trial163:Power off
2026-02-12T09:50:29.657 DEBUG:teuthology.orchestra.console.trial163:pexpect command: ipmitool -H trial163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T09:50:29.677 DEBUG:teuthology.orchestra.console.trial014:power off output: Chassis Power Control: Down/Off
2026-02-12T09:50:29.696 DEBUG:teuthology.orchestra.console.trial163:power off output: Chassis Power Control: Down/Off
2026-02-12T09:50:33.681 DEBUG:teuthology.orchestra.console.trial014:pexpect command: ipmitool -H trial014.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T09:50:33.697 DEBUG:teuthology.orchestra.console.trial163:pexpect command: ipmitool -H trial163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T09:50:33.791 DEBUG:teuthology.orchestra.console.trial014:check power output: Chassis Power is on
2026-02-12T09:50:33.805 DEBUG:teuthology.orchestra.console.trial163:check power output: Chassis Power is on
2026-02-12T09:50:37.795 DEBUG:teuthology.orchestra.console.trial014:pexpect command: ipmitool -H trial014.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T09:50:37.806 DEBUG:teuthology.orchestra.console.trial163:pexpect command: ipmitool -H trial163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T09:50:37.906 DEBUG:teuthology.orchestra.console.trial014:check power output: Chassis Power is off
2026-02-12T09:50:37.907 INFO:teuthology.orchestra.console.trial014:Power off completed
2026-02-12T09:50:37.917 DEBUG:teuthology.orchestra.console.trial163:check power output: Chassis Power is off
2026-02-12T09:50:37.917 INFO:teuthology.orchestra.console.trial163:Power off completed
2026-02-12T09:50:38.158 INFO:teuthology.lock.ops:unlocked: trial014.front.sepia.ceph.com
2026-02-12T09:50:38.172 INFO:teuthology.lock.ops:unlocked: trial163.front.sepia.ceph.com