2026-02-12T12:31:09.759 INFO:root:teuthology version: 0.0.1.dev29+gc433f1062
2026-02-12T12:31:09.759 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T12:31:09.832 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T12:31:10.021 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x773fb7e4d340>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x773fb76842c0>, signals=[15])
2026-02-12T12:31:10.021 INFO:teuthology.lock.ops:Start node 'trial046.front.sepia.ceph.com' reimaging
2026-02-12T12:31:10.021 INFO:teuthology.lock.ops:Updating [trial046.front.sepia.ceph.com]: reset os type and version on server
2026-02-12T12:31:10.021 INFO:teuthology.lock.ops:Updating trial046.front.sepia.ceph.com on lock server
2026-02-12T12:31:10.079 INFO:teuthology.lock.ops:Start node 'trial157.front.sepia.ceph.com' reimaging
2026-02-12T12:31:10.079 INFO:teuthology.lock.ops:Updating [trial157.front.sepia.ceph.com]: reset os type and version on server
2026-02-12T12:31:10.079 INFO:teuthology.lock.ops:Updating trial157.front.sepia.ceph.com on lock server
2026-02-12T12:31:10.286 DEBUG:teuthology.provision.fog.trial046:Requesting image trial_centos_9.stream (ID 3)
2026-02-12T12:31:10.297 DEBUG:teuthology.provision.fog.trial157:Requesting image trial_centos_9.stream (ID 3)
2026-02-12T12:31:10.340 INFO:teuthology.provision.fog.trial046:Scheduling deploy of centos 9.stream
2026-02-12T12:31:10.348 INFO:teuthology.provision.fog.trial157:Scheduling deploy of centos 9.stream
2026-02-12T12:31:11.510 INFO:teuthology.orchestra.console.trial046:Power off
2026-02-12T12:31:11.510 DEBUG:teuthology.orchestra.console.trial046:pexpect command: ipmitool -H trial046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T12:31:11.520 INFO:teuthology.orchestra.console.trial157:Power off
2026-02-12T12:31:11.520 DEBUG:teuthology.orchestra.console.trial157:pexpect command: ipmitool -H trial157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T12:31:11.548 DEBUG:teuthology.orchestra.console.trial046:power off output: Chassis Power Control: Down/Off
2026-02-12T12:31:11.557 DEBUG:teuthology.orchestra.console.trial157:power off output: Chassis Power Control: Down/Off
2026-02-12T12:31:15.553 DEBUG:teuthology.orchestra.console.trial046:pexpect command: ipmitool -H trial046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T12:31:15.560 DEBUG:teuthology.orchestra.console.trial157:pexpect command: ipmitool -H trial157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T12:31:15.663 DEBUG:teuthology.orchestra.console.trial046:check power output: Chassis Power is off
2026-02-12T12:31:15.663 INFO:teuthology.orchestra.console.trial046:Power off completed
2026-02-12T12:31:15.668 DEBUG:teuthology.orchestra.console.trial157:check power output: Chassis Power is off
2026-02-12T12:31:15.668 INFO:teuthology.orchestra.console.trial157:Power off completed
2026-02-12T12:31:15.764 INFO:teuthology.orchestra.console.trial046:Power on
2026-02-12T12:31:15.765 DEBUG:teuthology.orchestra.console.trial046:pexpect command: ipmitool -H trial046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-12T12:31:15.773 INFO:teuthology.orchestra.console.trial157:Power on
2026-02-12T12:31:15.773 DEBUG:teuthology.orchestra.console.trial157:pexpect command: ipmitool -H trial157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-12T12:31:15.805 DEBUG:teuthology.orchestra.console.trial046:power on output: Chassis Power Control: Up/On
2026-02-12T12:31:15.813 DEBUG:teuthology.orchestra.console.trial157:power on output: Chassis Power Control: Up/On
2026-02-12T12:31:19.810 DEBUG:teuthology.orchestra.console.trial046:pexpect command: ipmitool -H trial046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T12:31:19.819 DEBUG:teuthology.orchestra.console.trial157:pexpect command: ipmitool -H trial157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T12:31:19.921 DEBUG:teuthology.orchestra.console.trial046:check power output: Chassis Power is on
2026-02-12T12:31:19.922 INFO:teuthology.orchestra.console.trial046:Power on completed
2026-02-12T12:31:19.927 DEBUG:teuthology.orchestra.console.trial157:check power output: Chassis Power is off
2026-02-12T12:31:20.023 INFO:teuthology.provision.fog.trial046:Waiting for deploy to finish
2026-02-12T12:31:23.932 DEBUG:teuthology.orchestra.console.trial157:pexpect command: ipmitool -H trial157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T12:31:24.041 DEBUG:teuthology.orchestra.console.trial157:check power output: Chassis Power is on
2026-02-12T12:31:24.042 INFO:teuthology.orchestra.console.trial157:Power on completed
2026-02-12T12:31:24.143 INFO:teuthology.provision.fog.trial157:Waiting for deploy to finish
2026-02-12T12:33:06.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T12:33:10.045 WARNING:teuthology.provision.fog.trial046:[Errno None] Unable to connect to port 22 on 10.20.193.46
2026-02-12T12:33:10.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T12:33:14.014 WARNING:teuthology.provision.fog.trial157:[Errno None] Unable to connect to port 22 on 10.20.193.157
2026-02-12T12:33:16.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T12:33:19.133 WARNING:teuthology.provision.fog.trial046:[Errno None] Unable to connect to port 22 on 10.20.193.46
2026-02-12T12:33:20.017 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T12:33:23.101 WARNING:teuthology.provision.fog.trial157:[Errno None] Unable to connect to port 22 on 10.20.193.157
2026-02-12T12:33:25.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T12:33:28.221 WARNING:teuthology.provision.fog.trial046:[Errno None] Unable to connect to port 22 on 10.20.193.46
2026-02-12T12:33:29.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T12:33:32.189 WARNING:teuthology.provision.fog.trial157:[Errno None] Unable to connect to port 22 on 10.20.193.157
2026-02-12T12:33:34.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T12:33:37.309 WARNING:teuthology.provision.fog.trial046:[Errno None] Unable to connect to port 22 on 10.20.193.46
2026-02-12T12:33:38.190 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T12:33:41.277 WARNING:teuthology.provision.fog.trial157:[Errno None] Unable to connect to port 22 on 10.20.193.157
2026-02-12T12:33:43.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T12:33:47.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T12:34:19.552 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-12T12:34:19.553 DEBUG:teuthology.orchestra.run.trial046:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-12T12:34:23.141 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-12T12:34:23.141 DEBUG:teuthology.orchestra.run.trial157:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-12T12:34:23.301 INFO:teuthology.provision.fog.trial157:Node is ready
2026-02-12T12:34:23.301 DEBUG:teuthology.orchestra.run.trial157:> hostname
2026-02-12T12:34:23.357 INFO:teuthology.orchestra.run.trial157.stdout:trial157.front.sepia.ceph.com
2026-02-12T12:34:23.357 DEBUG:teuthology.orchestra.run.trial157:> grep trial157.front.sepia.ceph.com /etc/hosts
2026-02-12T12:34:23.413 INFO:teuthology.orchestra.run.trial157.stdout:10.20.193.157 trial157.front.sepia.ceph.com trial157
2026-02-12T12:34:23.414 DEBUG:teuthology.orchestra.run.trial157:> sudo hostname trial157
2026-02-12T12:34:23.483 DEBUG:teuthology.orchestra.run.trial157:> sudo sed -i -e 's/trial157.front.sepia.ceph.com/trial157/g' /etc/hosts
2026-02-12T12:34:23.547 DEBUG:teuthology.orchestra.run.trial157:> sudo sed -i -e 's/10.20.193.157/10.20.193.157/g' /etc/hosts
2026-02-12T12:34:23.612 DEBUG:teuthology.orchestra.run.trial157:> sudo sed -i -e 's/trial157.front.sepia.ceph.com/trial157/g' /etc/hostname
2026-02-12T12:34:23.676 DEBUG:teuthology.orchestra.run.trial157:> sudo hostname trial157
2026-02-12T12:34:23.740 DEBUG:teuthology.orchestra.run.trial157:> cat /etc/os-release
2026-02-12T12:34:23.796 INFO:teuthology.orchestra.run.trial157.stdout:NAME="CentOS Stream"
2026-02-12T12:34:23.796 INFO:teuthology.orchestra.run.trial157.stdout:VERSION="9"
2026-02-12T12:34:23.796 INFO:teuthology.orchestra.run.trial157.stdout:ID="centos"
2026-02-12T12:34:23.796 INFO:teuthology.orchestra.run.trial157.stdout:ID_LIKE="rhel fedora"
2026-02-12T12:34:23.796 INFO:teuthology.orchestra.run.trial157.stdout:VERSION_ID="9"
2026-02-12T12:34:23.796 INFO:teuthology.orchestra.run.trial157.stdout:PLATFORM_ID="platform:el9"
2026-02-12T12:34:23.796 INFO:teuthology.orchestra.run.trial157.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T12:34:23.797 INFO:teuthology.orchestra.run.trial157.stdout:ANSI_COLOR="0;31"
2026-02-12T12:34:23.797 INFO:teuthology.orchestra.run.trial157.stdout:LOGO="fedora-logo-icon"
2026-02-12T12:34:23.797 INFO:teuthology.orchestra.run.trial157.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T12:34:23.797 INFO:teuthology.orchestra.run.trial157.stdout:HOME_URL="https://centos.org/"
2026-02-12T12:34:23.797 INFO:teuthology.orchestra.run.trial157.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T12:34:23.797 INFO:teuthology.orchestra.run.trial157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T12:34:23.797 INFO:teuthology.orchestra.run.trial157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T12:34:23.798 INFO:teuthology.provision.fog.trial157:Deploy complete!
2026-02-12T12:34:23.799 DEBUG:teuthology.parallel:result is None
2026-02-12T12:34:25.192 INFO:teuthology.provision.fog.trial046:Node is ready
2026-02-12T12:34:25.192 DEBUG:teuthology.orchestra.run.trial046:> hostname
2026-02-12T12:34:25.210 INFO:teuthology.orchestra.run.trial046.stdout:trial046.front.sepia.ceph.com
2026-02-12T12:34:25.210 DEBUG:teuthology.orchestra.run.trial046:> grep trial046.front.sepia.ceph.com /etc/hosts
2026-02-12T12:34:25.270 INFO:teuthology.orchestra.run.trial046.stdout:10.20.193.46 trial046.front.sepia.ceph.com trial046
2026-02-12T12:34:25.270 DEBUG:teuthology.orchestra.run.trial046:> sudo hostname trial046
2026-02-12T12:34:25.343 DEBUG:teuthology.orchestra.run.trial046:> sudo sed -i -e 's/trial046.front.sepia.ceph.com/trial046/g' /etc/hosts
2026-02-12T12:34:25.412 DEBUG:teuthology.orchestra.run.trial046:> sudo sed -i -e 's/10.20.193.46/10.20.193.46/g' /etc/hosts
2026-02-12T12:34:25.480 DEBUG:teuthology.orchestra.run.trial046:> sudo sed -i -e 's/trial046.front.sepia.ceph.com/trial046/g' /etc/hostname
2026-02-12T12:34:25.548 DEBUG:teuthology.orchestra.run.trial046:> sudo hostname trial046
2026-02-12T12:34:25.615 DEBUG:teuthology.orchestra.run.trial046:> cat /etc/os-release
2026-02-12T12:34:25.675 INFO:teuthology.orchestra.run.trial046.stdout:NAME="CentOS Stream"
2026-02-12T12:34:25.675 INFO:teuthology.orchestra.run.trial046.stdout:VERSION="9"
2026-02-12T12:34:25.675 INFO:teuthology.orchestra.run.trial046.stdout:ID="centos"
2026-02-12T12:34:25.675 INFO:teuthology.orchestra.run.trial046.stdout:ID_LIKE="rhel fedora"
2026-02-12T12:34:25.676 INFO:teuthology.orchestra.run.trial046.stdout:VERSION_ID="9"
2026-02-12T12:34:25.676 INFO:teuthology.orchestra.run.trial046.stdout:PLATFORM_ID="platform:el9"
2026-02-12T12:34:25.676 INFO:teuthology.orchestra.run.trial046.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T12:34:25.676 INFO:teuthology.orchestra.run.trial046.stdout:ANSI_COLOR="0;31"
2026-02-12T12:34:25.676 INFO:teuthology.orchestra.run.trial046.stdout:LOGO="fedora-logo-icon"
2026-02-12T12:34:25.676 INFO:teuthology.orchestra.run.trial046.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T12:34:25.676 INFO:teuthology.orchestra.run.trial046.stdout:HOME_URL="https://centos.org/"
2026-02-12T12:34:25.676 INFO:teuthology.orchestra.run.trial046.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T12:34:25.676 INFO:teuthology.orchestra.run.trial046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T12:34:25.676 INFO:teuthology.orchestra.run.trial046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T12:34:25.677 INFO:teuthology.provision.fog.trial046:Deploy complete!
2026-02-12T12:34:25.678 DEBUG:teuthology.parallel:result is None
2026-02-12T12:34:26.090 INFO:teuthology.lock.ops:Checking trial046.front.sepia.ceph.com
2026-02-12T12:34:26.091 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-12T12:34:26.197 INFO:teuthology.lock.ops:Checking trial157.front.sepia.ceph.com
2026-02-12T12:34:26.197 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-12T12:34:26.261 INFO:teuthology.lock.ops:Updating [trial046.front.sepia.ceph.com]: set os type and version on server
2026-02-12T12:34:26.261 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial046.front.sepia.ceph.com'
2026-02-12T12:34:26.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T12:34:26.325 DEBUG:teuthology.orchestra.run.trial046:> true
2026-02-12T12:34:26.400 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial046.front.sepia.ceph.com'
2026-02-12T12:34:26.400 DEBUG:teuthology.orchestra.run.trial046:> uname -m
2026-02-12T12:34:26.459 INFO:teuthology.orchestra.run.trial046.stdout:x86_64
2026-02-12T12:34:26.459 DEBUG:teuthology.orchestra.run.trial046:> cat /etc/os-release
2026-02-12T12:34:26.516 INFO:teuthology.orchestra.run.trial046.stdout:NAME="CentOS Stream"
2026-02-12T12:34:26.516 INFO:teuthology.orchestra.run.trial046.stdout:VERSION="9"
2026-02-12T12:34:26.517 INFO:teuthology.orchestra.run.trial046.stdout:ID="centos"
2026-02-12T12:34:26.517 INFO:teuthology.orchestra.run.trial046.stdout:ID_LIKE="rhel fedora"
2026-02-12T12:34:26.517 INFO:teuthology.orchestra.run.trial046.stdout:VERSION_ID="9"
2026-02-12T12:34:26.517 INFO:teuthology.orchestra.run.trial046.stdout:PLATFORM_ID="platform:el9"
2026-02-12T12:34:26.517 INFO:teuthology.orchestra.run.trial046.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T12:34:26.517 INFO:teuthology.orchestra.run.trial046.stdout:ANSI_COLOR="0;31"
2026-02-12T12:34:26.517 INFO:teuthology.orchestra.run.trial046.stdout:LOGO="fedora-logo-icon"
2026-02-12T12:34:26.517 INFO:teuthology.orchestra.run.trial046.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T12:34:26.517 INFO:teuthology.orchestra.run.trial046.stdout:HOME_URL="https://centos.org/"
2026-02-12T12:34:26.517 INFO:teuthology.orchestra.run.trial046.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T12:34:26.517 INFO:teuthology.orchestra.run.trial046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T12:34:26.517 INFO:teuthology.orchestra.run.trial046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T12:34:26.518 INFO:teuthology.lock.ops:Updating trial046.front.sepia.ceph.com on lock server
2026-02-12T12:34:26.620 INFO:teuthology.lock.ops:Updating [trial157.front.sepia.ceph.com]: set os type and version on server
2026-02-12T12:34:26.620 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial157.front.sepia.ceph.com'
2026-02-12T12:34:26.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T12:34:26.692 DEBUG:teuthology.orchestra.run.trial157:> true
2026-02-12T12:34:26.768 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial157.front.sepia.ceph.com'
2026-02-12T12:34:26.768 DEBUG:teuthology.orchestra.run.trial157:> uname -m
2026-02-12T12:34:26.824 INFO:teuthology.orchestra.run.trial157.stdout:x86_64
2026-02-12T12:34:26.824 DEBUG:teuthology.orchestra.run.trial157:> cat /etc/os-release
2026-02-12T12:34:26.880 INFO:teuthology.orchestra.run.trial157.stdout:NAME="CentOS Stream"
2026-02-12T12:34:26.880 INFO:teuthology.orchestra.run.trial157.stdout:VERSION="9"
2026-02-12T12:34:26.881 INFO:teuthology.orchestra.run.trial157.stdout:ID="centos"
2026-02-12T12:34:26.881 INFO:teuthology.orchestra.run.trial157.stdout:ID_LIKE="rhel fedora"
2026-02-12T12:34:26.881 INFO:teuthology.orchestra.run.trial157.stdout:VERSION_ID="9"
2026-02-12T12:34:26.881 INFO:teuthology.orchestra.run.trial157.stdout:PLATFORM_ID="platform:el9"
2026-02-12T12:34:26.881 INFO:teuthology.orchestra.run.trial157.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T12:34:26.881 INFO:teuthology.orchestra.run.trial157.stdout:ANSI_COLOR="0;31"
2026-02-12T12:34:26.881 INFO:teuthology.orchestra.run.trial157.stdout:LOGO="fedora-logo-icon"
2026-02-12T12:34:26.881 INFO:teuthology.orchestra.run.trial157.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T12:34:26.881 INFO:teuthology.orchestra.run.trial157.stdout:HOME_URL="https://centos.org/"
2026-02-12T12:34:26.881 INFO:teuthology.orchestra.run.trial157.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T12:34:26.881 INFO:teuthology.orchestra.run.trial157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T12:34:26.881 INFO:teuthology.orchestra.run.trial157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T12:34:26.882 INFO:teuthology.lock.ops:Updating trial157.front.sepia.ceph.com on lock server
2026-02-12T12:34:26.984 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T12:34:27.083 INFO:teuthology.dispatcher.supervisor:Running job 42522
2026-02-12T12:34:27.083 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_c433f1062990a0488dc29a553589bc609a460691/virtualenv/bin/teuthology -v --owner scheduled_moagrawa@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/moagrawa-2026-02-10_06:35:43-crimson-rados-main-distro-debug-trial/42522 --name moagrawa-2026-02-10_06:35:43-crimson-rados-main-distro-debug-trial --description crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/default thrashosds-health workloads/snaps-few-objects-localized} -- /home/teuthworker/mnt/teuthology/moagrawa-2026-02-10_06:35:43-crimson-rados-main-distro-debug-trial/42522/orig.config.yaml
2026-02-12T12:34:27.084 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/moagrawa-2026-02-10_06:35:43-crimson-rados-main-distro-debug-trial/42522
2026-02-12T12:34:27.084 INFO:teuthology.dispatcher.supervisor:Job PID: 3855965
2026-02-12T12:34:27.084 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-12T12:36:27.143 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T12:38:27.362 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T12:40:27.552 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T12:42:27.667 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T12:44:27.850 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T12:46:28.037 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T12:48:28.195 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T12:50:28.381 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T12:52:28.595 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T12:54:28.821 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T12:56:29.015 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T12:58:29.197 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T13:00:29.384 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T13:02:29.594 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T13:04:29.767 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T13:06:29.952 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T13:08:30.147 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T13:10:30.304 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T13:12:30.467 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T13:14:30.663 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T13:16:30.870 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T13:18:31.085 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T13:20:31.272 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T13:22:31.453 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T13:24:31.622 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T13:26:31.749 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T13:28:31.959 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T13:28:32.055 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-12T13:28:32.171 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-12T13:28:32.680 INFO:teuthology.orchestra.console.trial046:Power off
2026-02-12T13:28:32.680 DEBUG:teuthology.orchestra.console.trial046:pexpect command: ipmitool -H trial046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T13:28:32.690 INFO:teuthology.orchestra.console.trial157:Power off
2026-02-12T13:28:32.690 DEBUG:teuthology.orchestra.console.trial157:pexpect command: ipmitool -H trial157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T13:28:32.720 DEBUG:teuthology.orchestra.console.trial046:power off output: Chassis Power Control: Down/Off
2026-02-12T13:28:32.725 DEBUG:teuthology.orchestra.console.trial157:power off output: Chassis Power Control: Down/Off
2026-02-12T13:28:36.725 DEBUG:teuthology.orchestra.console.trial046:pexpect command: ipmitool -H trial046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T13:28:36.731 DEBUG:teuthology.orchestra.console.trial157:pexpect command: ipmitool -H trial157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T13:28:36.833 DEBUG:teuthology.orchestra.console.trial046:check power output: Chassis Power is on
2026-02-12T13:28:36.838 DEBUG:teuthology.orchestra.console.trial157:check power output: Chassis Power is on
2026-02-12T13:28:40.838 DEBUG:teuthology.orchestra.console.trial046:pexpect command: ipmitool -H trial046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T13:28:40.846 DEBUG:teuthology.orchestra.console.trial157:pexpect command: ipmitool -H trial157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T13:28:40.948 DEBUG:teuthology.orchestra.console.trial046:check power output: Chassis Power is off
2026-02-12T13:28:40.948 INFO:teuthology.orchestra.console.trial046:Power off completed
2026-02-12T13:28:40.955 DEBUG:teuthology.orchestra.console.trial157:check power output: Chassis Power is off
2026-02-12T13:28:40.955 INFO:teuthology.orchestra.console.trial157:Power off completed
2026-02-12T13:28:41.195 INFO:teuthology.lock.ops:unlocked: trial046.front.sepia.ceph.com
2026-02-12T13:28:41.208 INFO:teuthology.lock.ops:unlocked: trial157.front.sepia.ceph.com