2026-02-12T09:03:07.175 INFO:root:teuthology version: 0.0.1.dev29+gc433f1062
2026-02-12T09:03:07.175 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:03:07.245 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:03:07.491 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x723405bf0ad0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x723405449260>, signals=[15])
2026-02-12T09:03:07.492 INFO:teuthology.lock.ops:Start node 'trial027.front.sepia.ceph.com' reimaging
2026-02-12T09:03:07.492 INFO:teuthology.lock.ops:Updating [trial027.front.sepia.ceph.com]: reset os type and version on server
2026-02-12T09:03:07.492 INFO:teuthology.lock.ops:Updating trial027.front.sepia.ceph.com on lock server
2026-02-12T09:03:07.549 INFO:teuthology.lock.ops:Start node 'trial056.front.sepia.ceph.com' reimaging
2026-02-12T09:03:07.549 INFO:teuthology.lock.ops:Updating [trial056.front.sepia.ceph.com]: reset os type and version on server
2026-02-12T09:03:07.549 INFO:teuthology.lock.ops:Updating trial056.front.sepia.ceph.com on lock server
2026-02-12T09:03:07.776 DEBUG:teuthology.provision.fog.trial056:Requesting image trial_centos_9.stream (ID 3)
2026-02-12T09:03:07.788 DEBUG:teuthology.provision.fog.trial027:Requesting image trial_centos_9.stream (ID 3)
2026-02-12T09:03:07.817 INFO:teuthology.provision.fog.trial056:Scheduling deploy of centos 9.stream
2026-02-12T09:03:07.834 INFO:teuthology.provision.fog.trial027:Scheduling deploy of centos 9.stream
2026-02-12T09:03:08.716 INFO:teuthology.orchestra.console.trial056:Power off
2026-02-12T09:03:08.716 DEBUG:teuthology.orchestra.console.trial056:pexpect command: ipmitool -H trial056.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T09:03:08.774 DEBUG:teuthology.orchestra.console.trial056:power off output: Chassis Power Control: Down/Off
2026-02-12T09:03:08.786 INFO:teuthology.orchestra.console.trial027:Power off
2026-02-12T09:03:08.786 DEBUG:teuthology.orchestra.console.trial027:pexpect command: ipmitool -H trial027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T09:03:08.823 DEBUG:teuthology.orchestra.console.trial027:power off output: Chassis Power Control: Down/Off
2026-02-12T09:03:12.779 DEBUG:teuthology.orchestra.console.trial056:pexpect command: ipmitool -H trial056.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T09:03:12.825 DEBUG:teuthology.orchestra.console.trial027:pexpect command: ipmitool -H trial027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T09:03:12.888 DEBUG:teuthology.orchestra.console.trial056:check power output: Chassis Power is off
2026-02-12T09:03:12.888 INFO:teuthology.orchestra.console.trial056:Power off completed
2026-02-12T09:03:12.932 DEBUG:teuthology.orchestra.console.trial027:check power output: Chassis Power is off
2026-02-12T09:03:12.932 INFO:teuthology.orchestra.console.trial027:Power off completed
2026-02-12T09:03:12.989 INFO:teuthology.orchestra.console.trial056:Power on
2026-02-12T09:03:12.989 DEBUG:teuthology.orchestra.console.trial056:pexpect command: ipmitool -H trial056.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-12T09:03:13.028 DEBUG:teuthology.orchestra.console.trial056:power on output: Chassis Power Control: Up/On
2026-02-12T09:03:13.033 INFO:teuthology.orchestra.console.trial027:Power on
2026-02-12T09:03:13.033 DEBUG:teuthology.orchestra.console.trial027:pexpect command: ipmitool -H trial027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-12T09:03:13.073 DEBUG:teuthology.orchestra.console.trial027:power on output: Chassis Power Control: Up/On
2026-02-12T09:03:17.030 DEBUG:teuthology.orchestra.console.trial056:pexpect command: ipmitool -H trial056.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T09:03:17.074 DEBUG:teuthology.orchestra.console.trial027:pexpect command: ipmitool -H trial027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T09:03:17.141 DEBUG:teuthology.orchestra.console.trial056:check power output: Chassis Power is on
2026-02-12T09:03:17.141 INFO:teuthology.orchestra.console.trial056:Power on completed
2026-02-12T09:03:17.183 DEBUG:teuthology.orchestra.console.trial027:check power output: Chassis Power is on
2026-02-12T09:03:17.183 INFO:teuthology.orchestra.console.trial027:Power on completed
2026-02-12T09:03:17.242 INFO:teuthology.provision.fog.trial056:Waiting for deploy to finish
2026-02-12T09:03:17.283 INFO:teuthology.provision.fog.trial027:Waiting for deploy to finish
2026-02-12T09:05:04.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial056.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:05:04.608 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:05:07.357 WARNING:teuthology.provision.fog.trial056:[Errno None] Unable to connect to port 22 on 10.20.193.56
2026-02-12T09:05:07.677 WARNING:teuthology.provision.fog.trial027:[Errno None] Unable to connect to port 22 on 10.20.193.27
2026-02-12T09:05:13.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial056.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:05:13.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:05:16.445 WARNING:teuthology.provision.fog.trial056:[Errno None] Unable to connect to port 22 on 10.20.193.56
2026-02-12T09:05:16.765 WARNING:teuthology.provision.fog.trial027:[Errno None] Unable to connect to port 22 on 10.20.193.27
2026-02-12T09:05:22.452 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial056.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:05:22.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:05:25.533 WARNING:teuthology.provision.fog.trial056:[Errno None] Unable to connect to port 22 on 10.20.193.56
2026-02-12T09:05:25.853 WARNING:teuthology.provision.fog.trial027:[Errno None] Unable to connect to port 22 on 10.20.193.27
2026-02-12T09:05:31.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial056.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:05:31.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:05:34.941 WARNING:teuthology.provision.fog.trial027:[Errno None] Unable to connect to port 22 on 10.20.193.27
2026-02-12T09:05:40.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:06:07.776 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-12T09:06:07.777 DEBUG:teuthology.orchestra.run.trial056:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-12T09:06:08.002 INFO:teuthology.provision.fog.trial056:Node is ready
2026-02-12T09:06:08.002 DEBUG:teuthology.orchestra.run.trial056:> hostname
2026-02-12T09:06:08.057 INFO:teuthology.orchestra.run.trial056.stdout:trial056.front.sepia.ceph.com
2026-02-12T09:06:08.058 DEBUG:teuthology.orchestra.run.trial056:> grep trial056.front.sepia.ceph.com /etc/hosts
2026-02-12T09:06:08.114 INFO:teuthology.orchestra.run.trial056.stdout:10.20.193.56 trial056.front.sepia.ceph.com trial056
2026-02-12T09:06:08.115 DEBUG:teuthology.orchestra.run.trial056:> sudo hostname trial056
2026-02-12T09:06:08.183 DEBUG:teuthology.orchestra.run.trial056:> sudo sed -i -e 's/trial056.front.sepia.ceph.com/trial056/g' /etc/hosts
2026-02-12T09:06:08.250 DEBUG:teuthology.orchestra.run.trial056:> sudo sed -i -e 's/10.20.193.56/10.20.193.56/g' /etc/hosts
2026-02-12T09:06:08.316 DEBUG:teuthology.orchestra.run.trial056:> sudo sed -i -e 's/trial056.front.sepia.ceph.com/trial056/g' /etc/hostname
2026-02-12T09:06:08.381 DEBUG:teuthology.orchestra.run.trial056:> sudo hostname trial056
2026-02-12T09:06:08.445 DEBUG:teuthology.orchestra.run.trial056:> cat /etc/os-release
2026-02-12T09:06:08.501 INFO:teuthology.orchestra.run.trial056.stdout:NAME="CentOS Stream"
2026-02-12T09:06:08.501 INFO:teuthology.orchestra.run.trial056.stdout:VERSION="9"
2026-02-12T09:06:08.501 INFO:teuthology.orchestra.run.trial056.stdout:ID="centos"
2026-02-12T09:06:08.501 INFO:teuthology.orchestra.run.trial056.stdout:ID_LIKE="rhel fedora"
2026-02-12T09:06:08.501 INFO:teuthology.orchestra.run.trial056.stdout:VERSION_ID="9"
2026-02-12T09:06:08.501 INFO:teuthology.orchestra.run.trial056.stdout:PLATFORM_ID="platform:el9"
2026-02-12T09:06:08.501 INFO:teuthology.orchestra.run.trial056.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T09:06:08.501 INFO:teuthology.orchestra.run.trial056.stdout:ANSI_COLOR="0;31"
2026-02-12T09:06:08.501 INFO:teuthology.orchestra.run.trial056.stdout:LOGO="fedora-logo-icon"
2026-02-12T09:06:08.502 INFO:teuthology.orchestra.run.trial056.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T09:06:08.502 INFO:teuthology.orchestra.run.trial056.stdout:HOME_URL="https://centos.org/"
2026-02-12T09:06:08.502 INFO:teuthology.orchestra.run.trial056.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T09:06:08.502 INFO:teuthology.orchestra.run.trial056.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T09:06:08.502 INFO:teuthology.orchestra.run.trial056.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T09:06:08.503 INFO:teuthology.provision.fog.trial056:Deploy complete!
2026-02-12T09:06:08.504 DEBUG:teuthology.parallel:result is None
2026-02-12T09:06:17.060 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-12T09:06:17.061 DEBUG:teuthology.orchestra.run.trial027:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-12T09:06:17.305 INFO:teuthology.provision.fog.trial027:Node is ready
2026-02-12T09:06:17.305 DEBUG:teuthology.orchestra.run.trial027:> hostname
2026-02-12T09:06:17.358 INFO:teuthology.orchestra.run.trial027.stdout:trial027.front.sepia.ceph.com
2026-02-12T09:06:17.359 DEBUG:teuthology.orchestra.run.trial027:> grep trial027.front.sepia.ceph.com /etc/hosts
2026-02-12T09:06:17.412 INFO:teuthology.orchestra.run.trial027.stdout:10.20.193.27 trial027.front.sepia.ceph.com trial027
2026-02-12T09:06:17.413 DEBUG:teuthology.orchestra.run.trial027:> sudo hostname trial027
2026-02-12T09:06:17.479 DEBUG:teuthology.orchestra.run.trial027:> sudo sed -i -e 's/trial027.front.sepia.ceph.com/trial027/g' /etc/hosts
2026-02-12T09:06:17.545 DEBUG:teuthology.orchestra.run.trial027:> sudo sed -i -e 's/10.20.193.27/10.20.193.27/g' /etc/hosts
2026-02-12T09:06:17.609 DEBUG:teuthology.orchestra.run.trial027:> sudo sed -i -e 's/trial027.front.sepia.ceph.com/trial027/g' /etc/hostname
2026-02-12T09:06:17.670 DEBUG:teuthology.orchestra.run.trial027:> sudo hostname trial027
2026-02-12T09:06:17.732 DEBUG:teuthology.orchestra.run.trial027:> cat /etc/os-release
2026-02-12T09:06:17.786 INFO:teuthology.orchestra.run.trial027.stdout:NAME="CentOS Stream"
2026-02-12T09:06:17.787 INFO:teuthology.orchestra.run.trial027.stdout:VERSION="9"
2026-02-12T09:06:17.787 INFO:teuthology.orchestra.run.trial027.stdout:ID="centos"
2026-02-12T09:06:17.787 INFO:teuthology.orchestra.run.trial027.stdout:ID_LIKE="rhel fedora"
2026-02-12T09:06:17.787 INFO:teuthology.orchestra.run.trial027.stdout:VERSION_ID="9"
2026-02-12T09:06:17.787 INFO:teuthology.orchestra.run.trial027.stdout:PLATFORM_ID="platform:el9"
2026-02-12T09:06:17.787 INFO:teuthology.orchestra.run.trial027.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T09:06:17.787 INFO:teuthology.orchestra.run.trial027.stdout:ANSI_COLOR="0;31"
2026-02-12T09:06:17.787 INFO:teuthology.orchestra.run.trial027.stdout:LOGO="fedora-logo-icon"
2026-02-12T09:06:17.787 INFO:teuthology.orchestra.run.trial027.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T09:06:17.787 INFO:teuthology.orchestra.run.trial027.stdout:HOME_URL="https://centos.org/"
2026-02-12T09:06:17.788 INFO:teuthology.orchestra.run.trial027.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T09:06:17.788 INFO:teuthology.orchestra.run.trial027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T09:06:17.788 INFO:teuthology.orchestra.run.trial027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T09:06:17.788 INFO:teuthology.provision.fog.trial027:Deploy complete!
2026-02-12T09:06:17.789 DEBUG:teuthology.parallel:result is None
2026-02-12T09:06:18.197 INFO:teuthology.lock.ops:Checking trial027.front.sepia.ceph.com
2026-02-12T09:06:18.197 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-12T09:06:18.292 INFO:teuthology.lock.ops:Checking trial056.front.sepia.ceph.com
2026-02-12T09:06:18.292 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-12T09:06:18.354 INFO:teuthology.lock.ops:Updating [trial027.front.sepia.ceph.com]: set os type and version on server
2026-02-12T09:06:18.354 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial027.front.sepia.ceph.com'
2026-02-12T09:06:18.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:06:18.414 DEBUG:teuthology.orchestra.run.trial027:> true
2026-02-12T09:06:18.505 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial027.front.sepia.ceph.com'
2026-02-12T09:06:18.505 DEBUG:teuthology.orchestra.run.trial027:> uname -m
2026-02-12T09:06:18.559 INFO:teuthology.orchestra.run.trial027.stdout:x86_64
2026-02-12T09:06:18.559 DEBUG:teuthology.orchestra.run.trial027:> cat /etc/os-release
2026-02-12T09:06:18.616 INFO:teuthology.orchestra.run.trial027.stdout:NAME="CentOS Stream"
2026-02-12T09:06:18.616 INFO:teuthology.orchestra.run.trial027.stdout:VERSION="9"
2026-02-12T09:06:18.616 INFO:teuthology.orchestra.run.trial027.stdout:ID="centos"
2026-02-12T09:06:18.616 INFO:teuthology.orchestra.run.trial027.stdout:ID_LIKE="rhel fedora"
2026-02-12T09:06:18.616 INFO:teuthology.orchestra.run.trial027.stdout:VERSION_ID="9"
2026-02-12T09:06:18.616 INFO:teuthology.orchestra.run.trial027.stdout:PLATFORM_ID="platform:el9"
2026-02-12T09:06:18.616 INFO:teuthology.orchestra.run.trial027.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T09:06:18.616 INFO:teuthology.orchestra.run.trial027.stdout:ANSI_COLOR="0;31"
2026-02-12T09:06:18.616 INFO:teuthology.orchestra.run.trial027.stdout:LOGO="fedora-logo-icon"
2026-02-12T09:06:18.617 INFO:teuthology.orchestra.run.trial027.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T09:06:18.617 INFO:teuthology.orchestra.run.trial027.stdout:HOME_URL="https://centos.org/"
2026-02-12T09:06:18.617 INFO:teuthology.orchestra.run.trial027.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T09:06:18.617 INFO:teuthology.orchestra.run.trial027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T09:06:18.617 INFO:teuthology.orchestra.run.trial027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T09:06:18.617 INFO:teuthology.lock.ops:Updating trial027.front.sepia.ceph.com on lock server
2026-02-12T09:06:18.719 INFO:teuthology.lock.ops:Updating [trial056.front.sepia.ceph.com]: set os type and version on server
2026-02-12T09:06:18.719 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial056.front.sepia.ceph.com'
2026-02-12T09:06:18.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial056.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T09:06:18.785 DEBUG:teuthology.orchestra.run.trial056:> true
2026-02-12T09:06:18.925 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial056.front.sepia.ceph.com'
2026-02-12T09:06:18.925 DEBUG:teuthology.orchestra.run.trial056:> uname -m
2026-02-12T09:06:18.981 INFO:teuthology.orchestra.run.trial056.stdout:x86_64
2026-02-12T09:06:18.981 DEBUG:teuthology.orchestra.run.trial056:> cat /etc/os-release
2026-02-12T09:06:19.037 INFO:teuthology.orchestra.run.trial056.stdout:NAME="CentOS Stream"
2026-02-12T09:06:19.037 INFO:teuthology.orchestra.run.trial056.stdout:VERSION="9"
2026-02-12T09:06:19.038 INFO:teuthology.orchestra.run.trial056.stdout:ID="centos"
2026-02-12T09:06:19.038 INFO:teuthology.orchestra.run.trial056.stdout:ID_LIKE="rhel fedora"
2026-02-12T09:06:19.038 INFO:teuthology.orchestra.run.trial056.stdout:VERSION_ID="9"
2026-02-12T09:06:19.038 INFO:teuthology.orchestra.run.trial056.stdout:PLATFORM_ID="platform:el9"
2026-02-12T09:06:19.038 INFO:teuthology.orchestra.run.trial056.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T09:06:19.038 INFO:teuthology.orchestra.run.trial056.stdout:ANSI_COLOR="0;31"
2026-02-12T09:06:19.038 INFO:teuthology.orchestra.run.trial056.stdout:LOGO="fedora-logo-icon"
2026-02-12T09:06:19.038 INFO:teuthology.orchestra.run.trial056.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T09:06:19.038 INFO:teuthology.orchestra.run.trial056.stdout:HOME_URL="https://centos.org/"
2026-02-12T09:06:19.038 INFO:teuthology.orchestra.run.trial056.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T09:06:19.039 INFO:teuthology.orchestra.run.trial056.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T09:06:19.039 INFO:teuthology.orchestra.run.trial056.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T09:06:19.039 INFO:teuthology.lock.ops:Updating trial056.front.sepia.ceph.com on lock server
2026-02-12T09:06:19.142 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:06:19.235 INFO:teuthology.dispatcher.supervisor:Running job 41615
2026-02-12T09:06:19.235 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_c433f1062990a0488dc29a553589bc609a460691/virtualenv/bin/teuthology -v --owner scheduled_matan@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41615 --name matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial --description crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/crimson_bluestore thrashers/{default} thrashosds-health workloads/pool-snaps-few-objects} -- /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41615/orig.config.yaml
2026-02-12T09:06:19.236 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41615
2026-02-12T09:06:19.236 INFO:teuthology.dispatcher.supervisor:Job PID: 3411598
2026-02-12T09:06:19.236 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-12T09:08:19.238 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:10:19.437 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:12:19.641 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:14:19.839 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:16:19.938 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:18:20.112 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:20:20.220 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:22:20.389 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:24:20.596 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:26:20.804 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:28:21.018 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:30:21.226 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:32:21.365 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:34:21.454 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:36:21.667 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:38:21.786 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:40:21.974 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:42:22.077 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:44:22.192 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:46:22.412 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:48:22.606 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:50:22.807 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:52:23.016 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:54:23.201 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:56:23.394 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T09:58:23.566 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T10:00:23.752 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T10:02:23.963 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T10:04:24.162 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T10:06:24.369 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T10:08:24.592 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T10:10:24.803 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T10:12:24.964 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T10:14:25.154 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T10:14:25.231 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-12T10:14:25.348 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-12T10:14:25.869 INFO:teuthology.orchestra.console.trial056:Power off
2026-02-12T10:14:25.869 DEBUG:teuthology.orchestra.console.trial056:pexpect command: ipmitool -H trial056.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T10:14:25.880 INFO:teuthology.orchestra.console.trial027:Power off
2026-02-12T10:14:25.880 DEBUG:teuthology.orchestra.console.trial027:pexpect command: ipmitool -H trial027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T10:14:25.907 DEBUG:teuthology.orchestra.console.trial056:power off output: Chassis Power Control: Down/Off
2026-02-12T10:14:25.918 DEBUG:teuthology.orchestra.console.trial027:power off output: Chassis Power Control: Down/Off
2026-02-12T10:14:29.911 DEBUG:teuthology.orchestra.console.trial056:pexpect command: ipmitool -H trial056.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T10:14:29.919 DEBUG:teuthology.orchestra.console.trial027:pexpect command: ipmitool -H trial027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T10:14:30.020 DEBUG:teuthology.orchestra.console.trial056:check power output: Chassis Power is on
2026-02-12T10:14:30.028 DEBUG:teuthology.orchestra.console.trial027:check power output: Chassis Power is on
2026-02-12T10:14:34.026 DEBUG:teuthology.orchestra.console.trial056:pexpect command: ipmitool -H trial056.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T10:14:34.034 DEBUG:teuthology.orchestra.console.trial027:pexpect command: ipmitool -H trial027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T10:14:34.137 DEBUG:teuthology.orchestra.console.trial056:check power output: Chassis Power is off
2026-02-12T10:14:34.137 INFO:teuthology.orchestra.console.trial056:Power off completed
2026-02-12T10:14:34.144 DEBUG:teuthology.orchestra.console.trial027:check power output: Chassis Power is off
2026-02-12T10:14:34.144 INFO:teuthology.orchestra.console.trial027:Power off completed
2026-02-12T10:14:34.386 INFO:teuthology.lock.ops:unlocked: trial056.front.sepia.ceph.com
2026-02-12T10:14:34.398 INFO:teuthology.lock.ops:unlocked: trial027.front.sepia.ceph.com