2026-02-23T12:28:54.212 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-23T12:28:54.213 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:28:54.282 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:28:54.477 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7d4d8dbb89e0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7d4d8d41de40>, signals=[15])
2026-02-23T12:28:54.477 INFO:teuthology.lock.ops:Start node 'trial094.front.sepia.ceph.com' reimaging
2026-02-23T12:28:54.477 INFO:teuthology.lock.ops:Updating [trial094.front.sepia.ceph.com]: reset os type and version on server
2026-02-23T12:28:54.477 INFO:teuthology.lock.ops:Updating trial094.front.sepia.ceph.com on lock server
2026-02-23T12:28:54.533 INFO:teuthology.lock.ops:Start node 'trial117.front.sepia.ceph.com' reimaging
2026-02-23T12:28:54.533 INFO:teuthology.lock.ops:Updating [trial117.front.sepia.ceph.com]: reset os type and version on server
2026-02-23T12:28:54.533 INFO:teuthology.lock.ops:Updating trial117.front.sepia.ceph.com on lock server
2026-02-23T12:28:54.762 DEBUG:teuthology.provision.fog.trial094:Requesting image trial_centos_9.stream (ID 3)
2026-02-23T12:28:54.764 DEBUG:teuthology.provision.fog.trial117:Requesting image trial_centos_9.stream (ID 3)
2026-02-23T12:28:54.785 INFO:teuthology.provision.fog.trial094:Scheduling deploy of centos 9.stream
2026-02-23T12:28:54.812 INFO:teuthology.provision.fog.trial117:Scheduling deploy of centos 9.stream
2026-02-23T12:28:57.758 INFO:teuthology.orchestra.console.trial117:Power off
2026-02-23T12:28:57.758 DEBUG:teuthology.orchestra.console.trial117:pexpect command: ipmitool -H trial117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-23T12:28:57.770 INFO:teuthology.orchestra.console.trial094:Power off
2026-02-23T12:28:57.770 DEBUG:teuthology.orchestra.console.trial094:pexpect command: ipmitool -H trial094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-23T12:28:57.797 DEBUG:teuthology.orchestra.console.trial117:power off output: Chassis Power Control: Down/Off
2026-02-23T12:28:57.807 DEBUG:teuthology.orchestra.console.trial094:power off output: Chassis Power Control: Down/Off
2026-02-23T12:29:01.798 DEBUG:teuthology.orchestra.console.trial117:pexpect command: ipmitool -H trial117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T12:29:01.808 DEBUG:teuthology.orchestra.console.trial094:pexpect command: ipmitool -H trial094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T12:29:01.908 DEBUG:teuthology.orchestra.console.trial117:check power output: Chassis Power is off
2026-02-23T12:29:01.908 INFO:teuthology.orchestra.console.trial117:Power off completed
2026-02-23T12:29:01.917 DEBUG:teuthology.orchestra.console.trial094:check power output: Chassis Power is off
2026-02-23T12:29:01.917 INFO:teuthology.orchestra.console.trial094:Power off completed
2026-02-23T12:29:02.009 INFO:teuthology.orchestra.console.trial117:Power on
2026-02-23T12:29:02.010 DEBUG:teuthology.orchestra.console.trial117:pexpect command: ipmitool -H trial117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-23T12:29:02.019 INFO:teuthology.orchestra.console.trial094:Power on
2026-02-23T12:29:02.019 DEBUG:teuthology.orchestra.console.trial094:pexpect command: ipmitool -H trial094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-23T12:29:02.050 DEBUG:teuthology.orchestra.console.trial117:power on output: Chassis Power Control: Up/On
2026-02-23T12:29:02.059 DEBUG:teuthology.orchestra.console.trial094:power on output: Chassis Power Control: Up/On
2026-02-23T12:29:06.055 DEBUG:teuthology.orchestra.console.trial117:pexpect command: ipmitool -H trial117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T12:29:06.061 DEBUG:teuthology.orchestra.console.trial094:pexpect command: ipmitool -H trial094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T12:29:06.163 DEBUG:teuthology.orchestra.console.trial117:check power output: Chassis Power is on
2026-02-23T12:29:06.163 INFO:teuthology.orchestra.console.trial117:Power on completed
2026-02-23T12:29:06.168 DEBUG:teuthology.orchestra.console.trial094:check power output: Chassis Power is off
2026-02-23T12:29:06.264 INFO:teuthology.provision.fog.trial117:Waiting for deploy to finish
2026-02-23T12:29:10.171 DEBUG:teuthology.orchestra.console.trial094:pexpect command: ipmitool -H trial094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T12:29:10.278 DEBUG:teuthology.orchestra.console.trial094:check power output: Chassis Power is on
2026-02-23T12:29:10.278 INFO:teuthology.orchestra.console.trial094:Power on completed
2026-02-23T12:29:10.379 INFO:teuthology.provision.fog.trial094:Waiting for deploy to finish
2026-02-23T12:31:34.298 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:31:37.408 WARNING:teuthology.provision.fog.trial117:[Errno None] Unable to connect to port 22 on 10.20.193.117
2026-02-23T12:31:43.414 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:31:46.496 WARNING:teuthology.provision.fog.trial117:[Errno None] Unable to connect to port 22 on 10.20.193.117
2026-02-23T12:31:52.503 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:31:55.584 WARNING:teuthology.provision.fog.trial117:[Errno None] Unable to connect to port 22 on 10.20.193.117
2026-02-23T12:32:01.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:32:04.672 WARNING:teuthology.provision.fog.trial117:[Errno None] Unable to connect to port 22 on 10.20.193.117
2026-02-23T12:32:08.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:32:10.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:32:11.520 WARNING:teuthology.provision.fog.trial094:[Errno None] Unable to connect to port 22 on 10.20.193.94
2026-02-23T12:32:17.527 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:32:20.607 WARNING:teuthology.provision.fog.trial094:[Errno None] Unable to connect to port 22 on 10.20.193.94
2026-02-23T12:32:26.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:32:29.696 WARNING:teuthology.provision.fog.trial094:[Errno None] Unable to connect to port 22 on 10.20.193.94
2026-02-23T12:32:35.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:32:38.783 WARNING:teuthology.provision.fog.trial094:[Errno None] Unable to connect to port 22 on 10.20.193.94
2026-02-23T12:32:44.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:32:46.552 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-23T12:32:46.552 DEBUG:teuthology.orchestra.run.trial117:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-23T12:32:47.871 WARNING:teuthology.provision.fog.trial094:[Errno None] Unable to connect to port 22 on 10.20.193.94
2026-02-23T12:32:51.730 INFO:teuthology.provision.fog.trial117:Node is ready
2026-02-23T12:32:51.731 DEBUG:teuthology.orchestra.run.trial117:> hostname
2026-02-23T12:32:51.748 INFO:teuthology.orchestra.run.trial117.stdout:trial117.front.sepia.ceph.com
2026-02-23T12:32:51.749 DEBUG:teuthology.orchestra.run.trial117:> grep trial117.front.sepia.ceph.com /etc/hosts
2026-02-23T12:32:51.806 INFO:teuthology.orchestra.run.trial117.stdout:10.20.193.117 trial117.front.sepia.ceph.com trial117
2026-02-23T12:32:51.806 DEBUG:teuthology.orchestra.run.trial117:> sudo hostname trial117
2026-02-23T12:32:51.877 DEBUG:teuthology.orchestra.run.trial117:> sudo sed -i -e 's/trial117.front.sepia.ceph.com/trial117/g' /etc/hosts
2026-02-23T12:32:51.943 DEBUG:teuthology.orchestra.run.trial117:> sudo sed -i -e 's/10.20.193.117/10.20.193.117/g' /etc/hosts
2026-02-23T12:32:52.006 DEBUG:teuthology.orchestra.run.trial117:> sudo sed -i -e 's/trial117.front.sepia.ceph.com/trial117/g' /etc/hostname
2026-02-23T12:32:52.071 DEBUG:teuthology.orchestra.run.trial117:> sudo hostname trial117
2026-02-23T12:32:52.134 DEBUG:teuthology.orchestra.run.trial117:> cat /etc/os-release
2026-02-23T12:32:52.191 INFO:teuthology.orchestra.run.trial117.stdout:NAME="CentOS Stream"
2026-02-23T12:32:52.191 INFO:teuthology.orchestra.run.trial117.stdout:VERSION="9"
2026-02-23T12:32:52.191 INFO:teuthology.orchestra.run.trial117.stdout:ID="centos"
2026-02-23T12:32:52.191 INFO:teuthology.orchestra.run.trial117.stdout:ID_LIKE="rhel fedora"
2026-02-23T12:32:52.191 INFO:teuthology.orchestra.run.trial117.stdout:VERSION_ID="9"
2026-02-23T12:32:52.192 INFO:teuthology.orchestra.run.trial117.stdout:PLATFORM_ID="platform:el9"
2026-02-23T12:32:52.192 INFO:teuthology.orchestra.run.trial117.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-23T12:32:52.192 INFO:teuthology.orchestra.run.trial117.stdout:ANSI_COLOR="0;31"
2026-02-23T12:32:52.192 INFO:teuthology.orchestra.run.trial117.stdout:LOGO="fedora-logo-icon"
2026-02-23T12:32:52.192 INFO:teuthology.orchestra.run.trial117.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-23T12:32:52.192 INFO:teuthology.orchestra.run.trial117.stdout:HOME_URL="https://centos.org/"
2026-02-23T12:32:52.192 INFO:teuthology.orchestra.run.trial117.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-23T12:32:52.192 INFO:teuthology.orchestra.run.trial117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-23T12:32:52.192 INFO:teuthology.orchestra.run.trial117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-23T12:32:52.193 INFO:teuthology.provision.fog.trial117:Deploy complete!
2026-02-23T12:32:52.194 DEBUG:teuthology.parallel:result is None
2026-02-23T12:32:53.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:33:30.058 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-23T12:33:30.058 DEBUG:teuthology.orchestra.run.trial094:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-23T12:33:30.220 INFO:teuthology.provision.fog.trial094:Node is ready
2026-02-23T12:33:30.220 DEBUG:teuthology.orchestra.run.trial094:> hostname
2026-02-23T12:33:30.276 INFO:teuthology.orchestra.run.trial094.stdout:trial094.front.sepia.ceph.com
2026-02-23T12:33:30.276 DEBUG:teuthology.orchestra.run.trial094:> grep trial094.front.sepia.ceph.com /etc/hosts
2026-02-23T12:33:30.331 INFO:teuthology.orchestra.run.trial094.stdout:10.20.193.94 trial094.front.sepia.ceph.com trial094
2026-02-23T12:33:30.331 DEBUG:teuthology.orchestra.run.trial094:> sudo hostname trial094
2026-02-23T12:33:30.401 DEBUG:teuthology.orchestra.run.trial094:> sudo sed -i -e 's/trial094.front.sepia.ceph.com/trial094/g' /etc/hosts
2026-02-23T12:33:30.464 DEBUG:teuthology.orchestra.run.trial094:> sudo sed -i -e 's/10.20.193.94/10.20.193.94/g' /etc/hosts
2026-02-23T12:33:30.529 DEBUG:teuthology.orchestra.run.trial094:> sudo sed -i -e 's/trial094.front.sepia.ceph.com/trial094/g' /etc/hostname
2026-02-23T12:33:30.592 DEBUG:teuthology.orchestra.run.trial094:> sudo hostname trial094
2026-02-23T12:33:30.656 DEBUG:teuthology.orchestra.run.trial094:> cat /etc/os-release
2026-02-23T12:33:30.712 INFO:teuthology.orchestra.run.trial094.stdout:NAME="CentOS Stream"
2026-02-23T12:33:30.712 INFO:teuthology.orchestra.run.trial094.stdout:VERSION="9"
2026-02-23T12:33:30.712 INFO:teuthology.orchestra.run.trial094.stdout:ID="centos"
2026-02-23T12:33:30.712 INFO:teuthology.orchestra.run.trial094.stdout:ID_LIKE="rhel fedora"
2026-02-23T12:33:30.712 INFO:teuthology.orchestra.run.trial094.stdout:VERSION_ID="9"
2026-02-23T12:33:30.712 INFO:teuthology.orchestra.run.trial094.stdout:PLATFORM_ID="platform:el9"
2026-02-23T12:33:30.712 INFO:teuthology.orchestra.run.trial094.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-23T12:33:30.712 INFO:teuthology.orchestra.run.trial094.stdout:ANSI_COLOR="0;31"
2026-02-23T12:33:30.712 INFO:teuthology.orchestra.run.trial094.stdout:LOGO="fedora-logo-icon"
2026-02-23T12:33:30.712 INFO:teuthology.orchestra.run.trial094.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-23T12:33:30.712 INFO:teuthology.orchestra.run.trial094.stdout:HOME_URL="https://centos.org/"
2026-02-23T12:33:30.712 INFO:teuthology.orchestra.run.trial094.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-23T12:33:30.712 INFO:teuthology.orchestra.run.trial094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-23T12:33:30.712 INFO:teuthology.orchestra.run.trial094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-23T12:33:30.712 INFO:teuthology.provision.fog.trial094:Deploy complete!
2026-02-23T12:33:30.713 DEBUG:teuthology.parallel:result is None
2026-02-23T12:33:31.061 INFO:teuthology.lock.ops:Checking trial094.front.sepia.ceph.com
2026-02-23T12:33:31.061 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-23T12:33:31.149 INFO:teuthology.lock.ops:Checking trial117.front.sepia.ceph.com
2026-02-23T12:33:31.149 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-23T12:33:31.210 INFO:teuthology.lock.ops:Updating [trial094.front.sepia.ceph.com]: set os type and version on server
2026-02-23T12:33:31.210 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial094.front.sepia.ceph.com'
2026-02-23T12:33:31.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:33:31.271 DEBUG:teuthology.orchestra.run.trial094:> true
2026-02-23T12:33:31.351 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial094.front.sepia.ceph.com'
2026-02-23T12:33:31.351 DEBUG:teuthology.orchestra.run.trial094:> uname -m
2026-02-23T12:33:31.406 INFO:teuthology.orchestra.run.trial094.stdout:x86_64
2026-02-23T12:33:31.406 DEBUG:teuthology.orchestra.run.trial094:> cat /etc/os-release
2026-02-23T12:33:31.462 INFO:teuthology.orchestra.run.trial094.stdout:NAME="CentOS Stream"
2026-02-23T12:33:31.462 INFO:teuthology.orchestra.run.trial094.stdout:VERSION="9"
2026-02-23T12:33:31.462 INFO:teuthology.orchestra.run.trial094.stdout:ID="centos"
2026-02-23T12:33:31.462 INFO:teuthology.orchestra.run.trial094.stdout:ID_LIKE="rhel fedora"
2026-02-23T12:33:31.462 INFO:teuthology.orchestra.run.trial094.stdout:VERSION_ID="9"
2026-02-23T12:33:31.462 INFO:teuthology.orchestra.run.trial094.stdout:PLATFORM_ID="platform:el9"
2026-02-23T12:33:31.462 INFO:teuthology.orchestra.run.trial094.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-23T12:33:31.462 INFO:teuthology.orchestra.run.trial094.stdout:ANSI_COLOR="0;31"
2026-02-23T12:33:31.462 INFO:teuthology.orchestra.run.trial094.stdout:LOGO="fedora-logo-icon"
2026-02-23T12:33:31.462 INFO:teuthology.orchestra.run.trial094.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-23T12:33:31.462 INFO:teuthology.orchestra.run.trial094.stdout:HOME_URL="https://centos.org/"
2026-02-23T12:33:31.462 INFO:teuthology.orchestra.run.trial094.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-23T12:33:31.462 INFO:teuthology.orchestra.run.trial094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-23T12:33:31.462 INFO:teuthology.orchestra.run.trial094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-23T12:33:31.463 INFO:teuthology.lock.ops:Updating trial094.front.sepia.ceph.com on lock server
2026-02-23T12:33:31.523 INFO:teuthology.lock.ops:Updating [trial117.front.sepia.ceph.com]: set os type and version on server
2026-02-23T12:33:31.523 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial117.front.sepia.ceph.com'
2026-02-23T12:33:31.524 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-23T12:33:31.585 DEBUG:teuthology.orchestra.run.trial117:> true
2026-02-23T12:33:31.659 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial117.front.sepia.ceph.com'
2026-02-23T12:33:31.659 DEBUG:teuthology.orchestra.run.trial117:> uname -m
2026-02-23T12:33:31.715 INFO:teuthology.orchestra.run.trial117.stdout:x86_64
2026-02-23T12:33:31.715 DEBUG:teuthology.orchestra.run.trial117:> cat /etc/os-release
2026-02-23T12:33:31.771 INFO:teuthology.orchestra.run.trial117.stdout:NAME="CentOS Stream"
2026-02-23T12:33:31.771 INFO:teuthology.orchestra.run.trial117.stdout:VERSION="9"
2026-02-23T12:33:31.771 INFO:teuthology.orchestra.run.trial117.stdout:ID="centos"
2026-02-23T12:33:31.771 INFO:teuthology.orchestra.run.trial117.stdout:ID_LIKE="rhel fedora"
2026-02-23T12:33:31.771 INFO:teuthology.orchestra.run.trial117.stdout:VERSION_ID="9"
2026-02-23T12:33:31.771 INFO:teuthology.orchestra.run.trial117.stdout:PLATFORM_ID="platform:el9"
2026-02-23T12:33:31.771 INFO:teuthology.orchestra.run.trial117.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-23T12:33:31.771 INFO:teuthology.orchestra.run.trial117.stdout:ANSI_COLOR="0;31"
2026-02-23T12:33:31.771 INFO:teuthology.orchestra.run.trial117.stdout:LOGO="fedora-logo-icon"
2026-02-23T12:33:31.771 INFO:teuthology.orchestra.run.trial117.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-23T12:33:31.771 INFO:teuthology.orchestra.run.trial117.stdout:HOME_URL="https://centos.org/"
2026-02-23T12:33:31.771 INFO:teuthology.orchestra.run.trial117.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-23T12:33:31.771 INFO:teuthology.orchestra.run.trial117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-23T12:33:31.771 INFO:teuthology.orchestra.run.trial117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-23T12:33:31.772 INFO:teuthology.lock.ops:Updating trial117.front.sepia.ceph.com on lock server
2026-02-23T12:33:31.845 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:33:31.937 INFO:teuthology.dispatcher.supervisor:Running job 66411
2026-02-23T12:33:31.937 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/bin/teuthology -v --owner scheduled_shraddhaag@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-23_12:27:41-crimson-rados-main-distro-debug-trial/66411 --name shraddhaag-2026-02-23_12:27:41-crimson-rados-main-distro-debug-trial --description crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/cephadm objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} thrashers/{default} thrashosds-health workloads/snaps-few-objects} -- /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-23_12:27:41-crimson-rados-main-distro-debug-trial/66411/orig.config.yaml
2026-02-23T12:33:31.937 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-23_12:27:41-crimson-rados-main-distro-debug-trial/66411
2026-02-23T12:33:31.937 INFO:teuthology.dispatcher.supervisor:Job PID: 570327
2026-02-23T12:33:31.937 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-23T12:35:32.036 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:37:32.159 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:39:32.359 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:41:32.548 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:43:32.756 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:45:32.963 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:47:33.169 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:49:33.375 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:51:33.580 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:53:33.722 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:55:33.916 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:57:34.083 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T12:59:34.290 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:01:34.467 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:03:34.655 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:05:34.841 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:07:35.021 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:09:35.191 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:11:35.379 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:13:35.571 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:15:35.695 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:17:35.865 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:19:36.048 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:21:36.250 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:23:36.436 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:25:36.601 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:27:36.785 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:29:36.973 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:31:37.168 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:33:39.792 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:35:39.948 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:37:40.317 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:39:40.404 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:41:40.576 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:43:40.790 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:45:41.008 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:47:41.585 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:49:41.758 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:51:41.952 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:53:42.159 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:55:42.367 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:57:42.575 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T13:59:42.733 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:01:42.942 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:03:43.151 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:05:43.359 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:07:43.549 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:09:43.754 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:11:43.959 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:13:44.174 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:15:44.382 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:17:44.543 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:19:44.753 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:21:44.953 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:23:45.154 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:25:45.350 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:27:45.555 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:29:45.925 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:31:46.147 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:33:46.360 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:35:46.579 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:37:46.768 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:39:46.969 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:41:47.165 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:43:47.360 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:45:47.569 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:47:47.775 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:49:47.988 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:51:48.197 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:53:48.406 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:55:48.575 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:57:48.770 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T14:59:48.976 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:01:49.126 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:03:49.332 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:05:49.540 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:07:49.750 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:09:49.961 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:11:50.088 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:13:50.310 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:15:50.467 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:17:50.589 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:19:50.790 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:21:50.997 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:23:51.226 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:25:51.436 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:27:51.642 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:29:51.870 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:31:52.080 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:33:52.288 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:35:52.666 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:37:52.880 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:39:53.087 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:41:53.297 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:43:53.506 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:45:53.730 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:47:53.944 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:49:54.156 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:51:54.369 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:53:54.578 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:55:54.785 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:57:54.995 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T15:59:55.203 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T16:01:55.416 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T16:03:55.623 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T16:05:55.829 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T16:07:56.078 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T16:09:56.287 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T16:11:56.405 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T16:13:58.665 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T16:15:58.874 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T16:17:59.081 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T16:19:59.294 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T16:22:01.173 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T16:24:01.387 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-23T16:24:01.477 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2026-02-23T16:24:01.593 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-23T16:24:02.084 INFO:teuthology.orchestra.console.trial094:Power off
2026-02-23T16:24:02.084 DEBUG:teuthology.orchestra.console.trial094:pexpect command: ipmitool -H trial094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-23T16:24:02.122 DEBUG:teuthology.orchestra.console.trial094:power off output: Chassis Power Control: Down/Off
2026-02-23T16:24:02.149 INFO:teuthology.orchestra.console.trial117:Power off
2026-02-23T16:24:02.149 DEBUG:teuthology.orchestra.console.trial117:pexpect command: ipmitool -H trial117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-23T16:24:02.186 DEBUG:teuthology.orchestra.console.trial117:power off output: Chassis Power Control: Down/Off
2026-02-23T16:24:06.127 DEBUG:teuthology.orchestra.console.trial094:pexpect command: ipmitool -H trial094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T16:24:06.187 DEBUG:teuthology.orchestra.console.trial117:pexpect command: ipmitool -H trial117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T16:24:06.236 DEBUG:teuthology.orchestra.console.trial094:check power output: Chassis Power is on
2026-02-23T16:24:06.295 DEBUG:teuthology.orchestra.console.trial117:check power output: Chassis Power is on
2026-02-23T16:24:10.242 DEBUG:teuthology.orchestra.console.trial094:pexpect command: ipmitool -H trial094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T16:24:10.296 DEBUG:teuthology.orchestra.console.trial117:pexpect command: ipmitool -H trial117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-23T16:24:10.352 DEBUG:teuthology.orchestra.console.trial094:check power output: Chassis Power is off
2026-02-23T16:24:10.352 INFO:teuthology.orchestra.console.trial094:Power off completed
2026-02-23T16:24:10.405 DEBUG:teuthology.orchestra.console.trial117:check power output: Chassis Power is off
2026-02-23T16:24:10.405 INFO:teuthology.orchestra.console.trial117:Power off completed
2026-02-23T16:24:10.600 INFO:teuthology.lock.ops:unlocked: trial094.front.sepia.ceph.com
2026-02-23T16:24:10.613 INFO:teuthology.lock.ops:unlocked: trial117.front.sepia.ceph.com