2026-02-23T12:29:02.769 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-23T12:29:02.769 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:29:02.842 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:29:03.029 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f8214161df0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f8213944720>, signals=[15])
2026-02-23T12:29:03.029 INFO:teuthology.lock.ops:Start node 'trial046.front.sepia.ceph.com' reimaging
2026-02-23T12:29:03.029 INFO:teuthology.lock.ops:Updating [trial046.front.sepia.ceph.com]: reset os type and version on server
2026-02-23T12:29:03.029 INFO:teuthology.lock.ops:Updating trial046.front.sepia.ceph.com on lock server
2026-02-23T12:29:03.085 INFO:teuthology.lock.ops:Start node 'trial131.front.sepia.ceph.com' reimaging
2026-02-23T12:29:03.085 INFO:teuthology.lock.ops:Updating [trial131.front.sepia.ceph.com]: reset os type and version on server
2026-02-23T12:29:03.085 INFO:teuthology.lock.ops:Updating trial131.front.sepia.ceph.com on lock server
2026-02-23T12:29:03.294 DEBUG:teuthology.provision.fog.trial131:Requesting image trial_centos_9.stream (ID 3)
2026-02-23T12:29:03.315 DEBUG:teuthology.provision.fog.trial046:Requesting image trial_centos_9.stream (ID 3)
2026-02-23T12:29:03.329 INFO:teuthology.provision.fog.trial131:Scheduling deploy of centos 9.stream
2026-02-23T12:29:03.362 INFO:teuthology.provision.fog.trial046:Scheduling deploy of centos 9.stream
2026-02-23T12:29:06.477 INFO:teuthology.orchestra.console.trial046:Power off
2026-02-23T12:29:06.477 DEBUG:teuthology.orchestra.console.trial046:pexpect command: ipmitool -H trial046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-23T12:29:06.488 INFO:teuthology.orchestra.console.trial131:Power off
2026-02-23T12:29:06.488 DEBUG:teuthology.orchestra.console.trial131:pexpect command: ipmitool -H trial131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-23T12:29:06.516 DEBUG:teuthology.orchestra.console.trial046:power off output: Chassis Power Control: Down/Off
2026-02-23T12:29:06.531 DEBUG:teuthology.orchestra.console.trial131:power off output: Chassis Power Control: Down/Off
2026-02-23T12:29:10.519 DEBUG:teuthology.orchestra.console.trial046:pexpect command: ipmitool -H trial046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T12:29:10.532 DEBUG:teuthology.orchestra.console.trial131:pexpect command: ipmitool -H trial131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T12:29:10.629 DEBUG:teuthology.orchestra.console.trial046:check power output: Chassis Power is off
2026-02-23T12:29:10.630 INFO:teuthology.orchestra.console.trial046:Power off completed
2026-02-23T12:29:10.640 DEBUG:teuthology.orchestra.console.trial131:check power output: Chassis Power is off
2026-02-23T12:29:10.641 INFO:teuthology.orchestra.console.trial131:Power off completed
2026-02-23T12:29:10.731 INFO:teuthology.orchestra.console.trial046:Power on
2026-02-23T12:29:10.731 DEBUG:teuthology.orchestra.console.trial046:pexpect command: ipmitool -H trial046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-23T12:29:10.742 INFO:teuthology.orchestra.console.trial131:Power on
2026-02-23T12:29:10.742 DEBUG:teuthology.orchestra.console.trial131:pexpect command: ipmitool -H trial131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-23T12:29:10.769 DEBUG:teuthology.orchestra.console.trial046:power on output: Chassis Power Control: Up/On
2026-02-23T12:29:10.783 DEBUG:teuthology.orchestra.console.trial131:power on output: Chassis Power Control: Up/On
2026-02-23T12:29:14.774 DEBUG:teuthology.orchestra.console.trial046:pexpect command: ipmitool -H trial046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T12:29:14.784 DEBUG:teuthology.orchestra.console.trial131:pexpect command: ipmitool -H trial131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T12:29:14.885 DEBUG:teuthology.orchestra.console.trial046:check power output: Chassis Power is on
2026-02-23T12:29:14.885 INFO:teuthology.orchestra.console.trial046:Power on completed
2026-02-23T12:29:14.892 DEBUG:teuthology.orchestra.console.trial131:check power output: Chassis Power is on
2026-02-23T12:29:14.892 INFO:teuthology.orchestra.console.trial131:Power on completed
2026-02-23T12:29:14.987 INFO:teuthology.provision.fog.trial046:Waiting for deploy to finish
2026-02-23T12:29:14.993 INFO:teuthology.provision.fog.trial131:Waiting for deploy to finish
2026-02-23T12:31:41.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:31:44.128 WARNING:teuthology.provision.fog.trial046:[Errno None] Unable to connect to port 22 on 10.20.193.46
2026-02-23T12:31:50.134 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:31:53.216 WARNING:teuthology.provision.fog.trial046:[Errno None] Unable to connect to port 22 on 10.20.193.46
2026-02-23T12:31:56.505 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:31:59.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:31:59.617 WARNING:teuthology.provision.fog.trial131:[Errno None] Unable to connect to port 22 on 10.20.193.131
2026-02-23T12:32:02.304 WARNING:teuthology.provision.fog.trial046:[Errno None] Unable to connect to port 22 on 10.20.193.46
2026-02-23T12:32:05.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:32:08.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:32:08.705 WARNING:teuthology.provision.fog.trial131:[Errno None] Unable to connect to port 22 on 10.20.193.131
2026-02-23T12:32:11.392 WARNING:teuthology.provision.fog.trial046:[Errno None] Unable to connect to port 22 on 10.20.193.46
2026-02-23T12:32:14.710 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:32:17.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:32:17.792 WARNING:teuthology.provision.fog.trial131:[Errno None] Unable to connect to port 22 on 10.20.193.131
2026-02-23T12:32:20.480 WARNING:teuthology.provision.fog.trial046:[Errno None] Unable to connect to port 22 on 10.20.193.46
2026-02-23T12:32:23.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:32:26.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:32:26.880 WARNING:teuthology.provision.fog.trial131:[Errno None] Unable to connect to port 22 on 10.20.193.131
2026-02-23T12:32:32.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:33:02.417 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-23T12:33:02.417 DEBUG:teuthology.orchestra.run.trial046:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-23T12:33:02.619 INFO:teuthology.provision.fog.trial046:Node is ready
2026-02-23T12:33:02.619 DEBUG:teuthology.orchestra.run.trial046:> hostname
2026-02-23T12:33:02.673 INFO:teuthology.orchestra.run.trial046.stdout:trial046.front.sepia.ceph.com
2026-02-23T12:33:02.673 DEBUG:teuthology.orchestra.run.trial046:> grep trial046.front.sepia.ceph.com /etc/hosts
2026-02-23T12:33:02.727 INFO:teuthology.orchestra.run.trial046.stdout:10.20.193.46 trial046.front.sepia.ceph.com trial046
2026-02-23T12:33:02.727 DEBUG:teuthology.orchestra.run.trial046:> sudo hostname trial046
2026-02-23T12:33:02.793 DEBUG:teuthology.orchestra.run.trial046:> sudo sed -i -e 's/trial046.front.sepia.ceph.com/trial046/g' /etc/hosts
2026-02-23T12:33:02.857 DEBUG:teuthology.orchestra.run.trial046:> sudo sed -i -e 's/10.20.193.46/10.20.193.46/g' /etc/hosts
2026-02-23T12:33:02.919 DEBUG:teuthology.orchestra.run.trial046:> sudo sed -i -e 's/trial046.front.sepia.ceph.com/trial046/g' /etc/hostname
2026-02-23T12:33:02.982 DEBUG:teuthology.orchestra.run.trial046:> sudo hostname trial046
2026-02-23T12:33:03.045 DEBUG:teuthology.orchestra.run.trial046:> cat /etc/os-release
2026-02-23T12:33:03.096 INFO:teuthology.orchestra.run.trial046.stdout:NAME="CentOS Stream"
2026-02-23T12:33:03.096 INFO:teuthology.orchestra.run.trial046.stdout:VERSION="9"
2026-02-23T12:33:03.096 INFO:teuthology.orchestra.run.trial046.stdout:ID="centos"
2026-02-23T12:33:03.096 INFO:teuthology.orchestra.run.trial046.stdout:ID_LIKE="rhel fedora"
2026-02-23T12:33:03.096 INFO:teuthology.orchestra.run.trial046.stdout:VERSION_ID="9"
2026-02-23T12:33:03.096 INFO:teuthology.orchestra.run.trial046.stdout:PLATFORM_ID="platform:el9"
2026-02-23T12:33:03.096 INFO:teuthology.orchestra.run.trial046.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-23T12:33:03.096 INFO:teuthology.orchestra.run.trial046.stdout:ANSI_COLOR="0;31"
2026-02-23T12:33:03.096 INFO:teuthology.orchestra.run.trial046.stdout:LOGO="fedora-logo-icon"
2026-02-23T12:33:03.096 INFO:teuthology.orchestra.run.trial046.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-23T12:33:03.096 INFO:teuthology.orchestra.run.trial046.stdout:HOME_URL="https://centos.org/"
2026-02-23T12:33:03.096 INFO:teuthology.orchestra.run.trial046.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-23T12:33:03.096 INFO:teuthology.orchestra.run.trial046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-23T12:33:03.096 INFO:teuthology.orchestra.run.trial046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-23T12:33:03.096 INFO:teuthology.provision.fog.trial046:Deploy complete!
2026-02-23T12:33:03.097 DEBUG:teuthology.parallel:result is None
2026-02-23T12:33:09.074 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-23T12:33:09.074 DEBUG:teuthology.orchestra.run.trial131:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-23T12:33:09.237 INFO:teuthology.provision.fog.trial131:Node is ready
2026-02-23T12:33:09.237 DEBUG:teuthology.orchestra.run.trial131:> hostname
2026-02-23T12:33:09.296 INFO:teuthology.orchestra.run.trial131.stdout:trial131.front.sepia.ceph.com
2026-02-23T12:33:09.296 DEBUG:teuthology.orchestra.run.trial131:> grep trial131.front.sepia.ceph.com /etc/hosts
2026-02-23T12:33:09.356 INFO:teuthology.orchestra.run.trial131.stdout:10.20.193.131 trial131.front.sepia.ceph.com trial131
2026-02-23T12:33:09.356 DEBUG:teuthology.orchestra.run.trial131:> sudo hostname trial131
2026-02-23T12:33:09.429 DEBUG:teuthology.orchestra.run.trial131:> sudo sed -i -e 's/trial131.front.sepia.ceph.com/trial131/g' /etc/hosts
2026-02-23T12:33:09.497 DEBUG:teuthology.orchestra.run.trial131:> sudo sed -i -e 's/10.20.193.131/10.20.193.131/g' /etc/hosts
2026-02-23T12:33:09.566 DEBUG:teuthology.orchestra.run.trial131:> sudo sed -i -e 's/trial131.front.sepia.ceph.com/trial131/g' /etc/hostname
2026-02-23T12:33:09.634 DEBUG:teuthology.orchestra.run.trial131:> sudo hostname trial131
2026-02-23T12:33:09.701 DEBUG:teuthology.orchestra.run.trial131:> cat /etc/os-release
2026-02-23T12:33:09.760 INFO:teuthology.orchestra.run.trial131.stdout:NAME="CentOS Stream"
2026-02-23T12:33:09.761 INFO:teuthology.orchestra.run.trial131.stdout:VERSION="9"
2026-02-23T12:33:09.761 INFO:teuthology.orchestra.run.trial131.stdout:ID="centos"
2026-02-23T12:33:09.761 INFO:teuthology.orchestra.run.trial131.stdout:ID_LIKE="rhel fedora"
2026-02-23T12:33:09.761 INFO:teuthology.orchestra.run.trial131.stdout:VERSION_ID="9"
2026-02-23T12:33:09.761 INFO:teuthology.orchestra.run.trial131.stdout:PLATFORM_ID="platform:el9"
2026-02-23T12:33:09.761 INFO:teuthology.orchestra.run.trial131.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-23T12:33:09.762 INFO:teuthology.orchestra.run.trial131.stdout:ANSI_COLOR="0;31"
2026-02-23T12:33:09.762 INFO:teuthology.orchestra.run.trial131.stdout:LOGO="fedora-logo-icon"
2026-02-23T12:33:09.762 INFO:teuthology.orchestra.run.trial131.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-23T12:33:09.762 INFO:teuthology.orchestra.run.trial131.stdout:HOME_URL="https://centos.org/"
2026-02-23T12:33:09.762 INFO:teuthology.orchestra.run.trial131.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-23T12:33:09.762 INFO:teuthology.orchestra.run.trial131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-23T12:33:09.762 INFO:teuthology.orchestra.run.trial131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-23T12:33:09.763 INFO:teuthology.provision.fog.trial131:Deploy complete!
2026-02-23T12:33:09.763 DEBUG:teuthology.parallel:result is None
2026-02-23T12:33:10.117 INFO:teuthology.lock.ops:Checking trial046.front.sepia.ceph.com
2026-02-23T12:33:10.117 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-23T12:33:10.182 INFO:teuthology.lock.ops:Checking trial131.front.sepia.ceph.com
2026-02-23T12:33:10.182 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-23T12:33:10.239 INFO:teuthology.lock.ops:Updating [trial046.front.sepia.ceph.com]: set os type and version on server
2026-02-23T12:33:10.240 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial046.front.sepia.ceph.com'
2026-02-23T12:33:10.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:33:10.303 DEBUG:teuthology.orchestra.run.trial046:> true
2026-02-23T12:33:10.376 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial046.front.sepia.ceph.com'
2026-02-23T12:33:10.376 DEBUG:teuthology.orchestra.run.trial046:> uname -m
2026-02-23T12:33:10.432 INFO:teuthology.orchestra.run.trial046.stdout:x86_64
2026-02-23T12:33:10.432 DEBUG:teuthology.orchestra.run.trial046:> cat /etc/os-release
2026-02-23T12:33:10.488 INFO:teuthology.orchestra.run.trial046.stdout:NAME="CentOS Stream"
2026-02-23T12:33:10.488 INFO:teuthology.orchestra.run.trial046.stdout:VERSION="9"
2026-02-23T12:33:10.488 INFO:teuthology.orchestra.run.trial046.stdout:ID="centos"
2026-02-23T12:33:10.488 INFO:teuthology.orchestra.run.trial046.stdout:ID_LIKE="rhel fedora"
2026-02-23T12:33:10.488 INFO:teuthology.orchestra.run.trial046.stdout:VERSION_ID="9"
2026-02-23T12:33:10.488 INFO:teuthology.orchestra.run.trial046.stdout:PLATFORM_ID="platform:el9"
2026-02-23T12:33:10.488 INFO:teuthology.orchestra.run.trial046.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-23T12:33:10.488 INFO:teuthology.orchestra.run.trial046.stdout:ANSI_COLOR="0;31"
2026-02-23T12:33:10.488 INFO:teuthology.orchestra.run.trial046.stdout:LOGO="fedora-logo-icon"
2026-02-23T12:33:10.488 INFO:teuthology.orchestra.run.trial046.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-23T12:33:10.489 INFO:teuthology.orchestra.run.trial046.stdout:HOME_URL="https://centos.org/"
2026-02-23T12:33:10.489 INFO:teuthology.orchestra.run.trial046.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-23T12:33:10.489 INFO:teuthology.orchestra.run.trial046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-23T12:33:10.489 INFO:teuthology.orchestra.run.trial046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-23T12:33:10.489 INFO:teuthology.lock.ops:Updating trial046.front.sepia.ceph.com on lock server
2026-02-23T12:33:10.573 INFO:teuthology.lock.ops:Updating [trial131.front.sepia.ceph.com]: set os type and version on server
2026-02-23T12:33:10.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial131.front.sepia.ceph.com'
2026-02-23T12:33:10.573 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:33:10.636 DEBUG:teuthology.orchestra.run.trial131:> true
2026-02-23T12:33:10.709 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial131.front.sepia.ceph.com'
2026-02-23T12:33:10.709 DEBUG:teuthology.orchestra.run.trial131:> uname -m
2026-02-23T12:33:10.764 INFO:teuthology.orchestra.run.trial131.stdout:x86_64
2026-02-23T12:33:10.765 DEBUG:teuthology.orchestra.run.trial131:> cat /etc/os-release
2026-02-23T12:33:10.820 INFO:teuthology.orchestra.run.trial131.stdout:NAME="CentOS Stream"
2026-02-23T12:33:10.820 INFO:teuthology.orchestra.run.trial131.stdout:VERSION="9"
2026-02-23T12:33:10.820 INFO:teuthology.orchestra.run.trial131.stdout:ID="centos"
2026-02-23T12:33:10.820 INFO:teuthology.orchestra.run.trial131.stdout:ID_LIKE="rhel fedora"
2026-02-23T12:33:10.820 INFO:teuthology.orchestra.run.trial131.stdout:VERSION_ID="9"
2026-02-23T12:33:10.820 INFO:teuthology.orchestra.run.trial131.stdout:PLATFORM_ID="platform:el9"
2026-02-23T12:33:10.820 INFO:teuthology.orchestra.run.trial131.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-23T12:33:10.820 INFO:teuthology.orchestra.run.trial131.stdout:ANSI_COLOR="0;31"
2026-02-23T12:33:10.820 INFO:teuthology.orchestra.run.trial131.stdout:LOGO="fedora-logo-icon"
2026-02-23T12:33:10.820 INFO:teuthology.orchestra.run.trial131.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-23T12:33:10.820 INFO:teuthology.orchestra.run.trial131.stdout:HOME_URL="https://centos.org/"
2026-02-23T12:33:10.820 INFO:teuthology.orchestra.run.trial131.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-23T12:33:10.820 INFO:teuthology.orchestra.run.trial131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-23T12:33:10.820 INFO:teuthology.orchestra.run.trial131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-23T12:33:10.821 INFO:teuthology.lock.ops:Updating trial131.front.sepia.ceph.com on lock server
2026-02-23T12:33:10.890 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:33:10.979 INFO:teuthology.dispatcher.supervisor:Running job 66427
2026-02-23T12:33:10.979 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/bin/teuthology -v --owner scheduled_shraddhaag@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-23_12:27:41-crimson-rados-main-distro-debug-trial/66427 --name shraddhaag-2026-02-23_12:27:41-crimson-rados-main-distro-debug-trial --description crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/cephadm objectstore/seastore/segmented$/{crimson_seastore_segmented} thrashers/{pggrow} thrashosds-health workloads/radosbench} -- /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-23_12:27:41-crimson-rados-main-distro-debug-trial/66427/orig.config.yaml
2026-02-23T12:33:10.979 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-23_12:27:41-crimson-rados-main-distro-debug-trial/66427
2026-02-23T12:33:10.979 INFO:teuthology.dispatcher.supervisor:Job PID: 565044
2026-02-23T12:33:10.979 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-23T12:35:11.079 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:37:11.248 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:39:11.456 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:41:11.622 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:43:11.764 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:45:11.946 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:47:12.077 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:49:12.232 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:51:12.390 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:53:12.548 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:55:12.724 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:57:12.922 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:59:13.112 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:01:13.291 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:03:13.472 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:05:13.592 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:07:13.780 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:09:13.966 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:11:14.161 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:13:14.362 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:15:14.544 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:17:14.689 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:19:14.853 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:21:15.341 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:23:23.024 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:25:23.197 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:27:23.347 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:29:23.547 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:31:23.722 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:33:23.917 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:35:24.237 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:37:24.410 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:39:24.609 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:41:24.781 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:43:24.931 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:43:25.023 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-23T13:43:25.166 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-23T13:43:25.691 INFO:teuthology.orchestra.console.trial046:Power off
2026-02-23T13:43:25.691 DEBUG:teuthology.orchestra.console.trial046:pexpect command: ipmitool -H trial046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-23T13:43:25.702 INFO:teuthology.orchestra.console.trial131:Power off
2026-02-23T13:43:25.702 DEBUG:teuthology.orchestra.console.trial131:pexpect command: ipmitool -H trial131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-23T13:43:25.729 DEBUG:teuthology.orchestra.console.trial046:power off output: Chassis Power Control: Down/Off
2026-02-23T13:43:25.739 DEBUG:teuthology.orchestra.console.trial131:power off output: Chassis Power Control: Down/Off
2026-02-23T13:43:29.734 DEBUG:teuthology.orchestra.console.trial046:pexpect command: ipmitool -H trial046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T13:43:29.741 DEBUG:teuthology.orchestra.console.trial131:pexpect command: ipmitool -H trial131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T13:43:29.844 DEBUG:teuthology.orchestra.console.trial046:check power output: Chassis Power is on
2026-02-23T13:43:29.850 DEBUG:teuthology.orchestra.console.trial131:check power output: Chassis Power is on
2026-02-23T13:43:33.850 DEBUG:teuthology.orchestra.console.trial046:pexpect command: ipmitool -H trial046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T13:43:33.861 DEBUG:teuthology.orchestra.console.trial131:pexpect command: ipmitool -H trial131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T13:43:33.962 DEBUG:teuthology.orchestra.console.trial046:check power output: Chassis Power is off
2026-02-23T13:43:33.962 INFO:teuthology.orchestra.console.trial046:Power off completed
2026-02-23T13:43:33.973 DEBUG:teuthology.orchestra.console.trial131:check power output: Chassis Power is off
2026-02-23T13:43:33.973 INFO:teuthology.orchestra.console.trial131:Power off completed
2026-02-23T13:43:34.236 INFO:teuthology.lock.ops:unlocked: trial046.front.sepia.ceph.com
2026-02-23T13:43:34.248 INFO:teuthology.lock.ops:unlocked: trial131.front.sepia.ceph.com