2026-02-25T09:59:10.526 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-25T09:59:10.526 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T09:59:10.600 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T09:59:10.794 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x72a2a0b86480>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x72a2a05c8400>, signals=[15])
2026-02-25T09:59:10.794 INFO:teuthology.lock.ops:Start node 'trial167.front.sepia.ceph.com' reimaging
2026-02-25T09:59:10.794 INFO:teuthology.lock.ops:Updating [trial167.front.sepia.ceph.com]: reset os type and version on server
2026-02-25T09:59:10.794 INFO:teuthology.lock.ops:Updating trial167.front.sepia.ceph.com on lock server
2026-02-25T09:59:10.853 INFO:teuthology.lock.ops:Start node 'trial190.front.sepia.ceph.com' reimaging
2026-02-25T09:59:10.853 INFO:teuthology.lock.ops:Updating [trial190.front.sepia.ceph.com]: reset os type and version on server
2026-02-25T09:59:10.853 INFO:teuthology.lock.ops:Updating trial190.front.sepia.ceph.com on lock server
2026-02-25T09:59:11.086 DEBUG:teuthology.provision.fog.trial167:Requesting image trial_centos_9.stream (ID 3)
2026-02-25T09:59:11.144 INFO:teuthology.provision.fog.trial167:Scheduling deploy of centos 9.stream
2026-02-25T09:59:11.226 DEBUG:teuthology.provision.fog.trial190:Requesting image trial_centos_9.stream (ID 3)
2026-02-25T09:59:11.279 INFO:teuthology.provision.fog.trial190:Scheduling deploy of centos 9.stream
2026-02-25T09:59:12.838 INFO:teuthology.orchestra.console.trial167:Power off
2026-02-25T09:59:12.838 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-25T09:59:12.849 INFO:teuthology.orchestra.console.trial190:Power off
2026-02-25T09:59:12.850 DEBUG:teuthology.orchestra.console.trial190:pexpect command: ipmitool -H trial190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-25T09:59:12.880 DEBUG:teuthology.orchestra.console.trial167:power off output: Chassis Power Control: Down/Off
2026-02-25T09:59:12.886 DEBUG:teuthology.orchestra.console.trial190:power off output: Chassis Power Control: Down/Off
2026-02-25T09:59:16.884 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T09:59:16.892 DEBUG:teuthology.orchestra.console.trial190:pexpect command: ipmitool -H trial190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T09:59:16.995 DEBUG:teuthology.orchestra.console.trial167:check power output: Chassis Power is off
2026-02-25T09:59:16.995 INFO:teuthology.orchestra.console.trial167:Power off completed
2026-02-25T09:59:17.000 DEBUG:teuthology.orchestra.console.trial190:check power output: Chassis Power is off
2026-02-25T09:59:17.000 INFO:teuthology.orchestra.console.trial190:Power off completed
2026-02-25T09:59:17.097 INFO:teuthology.orchestra.console.trial167:Power on
2026-02-25T09:59:17.097 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-25T09:59:17.106 INFO:teuthology.orchestra.console.trial190:Power on
2026-02-25T09:59:17.106 DEBUG:teuthology.orchestra.console.trial190:pexpect command: ipmitool -H trial190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-25T09:59:17.137 DEBUG:teuthology.orchestra.console.trial167:power on output: Chassis Power Control: Up/On
2026-02-25T09:59:17.146 DEBUG:teuthology.orchestra.console.trial190:power on output: Chassis Power Control: Up/On
2026-02-25T09:59:21.141 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T09:59:21.148 DEBUG:teuthology.orchestra.console.trial190:pexpect command: ipmitool -H trial190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T09:59:21.251 DEBUG:teuthology.orchestra.console.trial167:check power output: Chassis Power is on
2026-02-25T09:59:21.251 INFO:teuthology.orchestra.console.trial167:Power on completed
2026-02-25T09:59:21.255 DEBUG:teuthology.orchestra.console.trial190:check power output: Chassis Power is off
2026-02-25T09:59:21.352 INFO:teuthology.provision.fog.trial167:Waiting for deploy to finish
2026-02-25T09:59:25.259 DEBUG:teuthology.orchestra.console.trial190:pexpect command: ipmitool -H trial190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T09:59:25.370 DEBUG:teuthology.orchestra.console.trial190:check power output: Chassis Power is on
2026-02-25T09:59:25.370 INFO:teuthology.orchestra.console.trial190:Power on completed
2026-02-25T09:59:25.470 INFO:teuthology.provision.fog.trial190:Waiting for deploy to finish
2026-02-25T10:01:09.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T10:01:09.700 ERROR:teuthology.orchestra.connection:Error authenticating with trial167.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2026-02-25T10:01:12.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T10:01:15.968 WARNING:teuthology.provision.fog.trial190:[Errno None] Unable to connect to port 22 on 10.20.193.190
2026-02-25T10:01:21.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T10:01:25.056 WARNING:teuthology.provision.fog.trial190:[Errno None] Unable to connect to port 22 on 10.20.193.190
2026-02-25T10:01:31.056 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T10:01:34.143 WARNING:teuthology.provision.fog.trial190:[Errno None] Unable to connect to port 22 on 10.20.193.190
2026-02-25T10:01:40.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T10:01:43.232 WARNING:teuthology.provision.fog.trial190:[Errno None] Unable to connect to port 22 on 10.20.193.190
2026-02-25T10:01:49.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T10:01:52.320 WARNING:teuthology.provision.fog.trial190:[Errno None] Unable to connect to port 22 on 10.20.193.190
2026-02-25T10:01:58.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T10:02:13.703 WARNING:teuthology.provision.fog.trial167:timed out
2026-02-25T10:02:19.705 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T10:02:21.894 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-25T10:02:21.895 DEBUG:teuthology.orchestra.run.trial167:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-25T10:02:31.826 INFO:teuthology.provision.fog.trial167:Node is ready
2026-02-25T10:02:31.827 DEBUG:teuthology.orchestra.run.trial167:> hostname
2026-02-25T10:02:31.845 INFO:teuthology.orchestra.run.trial167.stdout:trial167.front.sepia.ceph.com
2026-02-25T10:02:31.845 DEBUG:teuthology.orchestra.run.trial167:> grep trial167.front.sepia.ceph.com /etc/hosts
2026-02-25T10:02:31.898 INFO:teuthology.orchestra.run.trial167.stdout:10.20.193.167 trial167.front.sepia.ceph.com trial167
2026-02-25T10:02:31.898 DEBUG:teuthology.orchestra.run.trial167:> sudo hostname trial167
2026-02-25T10:02:31.967 DEBUG:teuthology.orchestra.run.trial167:> sudo sed -i -e 's/trial167.front.sepia.ceph.com/trial167/g' /etc/hosts
2026-02-25T10:02:32.030 DEBUG:teuthology.orchestra.run.trial167:> sudo sed -i -e 's/10.20.193.167/10.20.193.167/g' /etc/hosts
2026-02-25T10:02:32.092 DEBUG:teuthology.orchestra.run.trial167:> sudo sed -i -e 's/trial167.front.sepia.ceph.com/trial167/g' /etc/hostname
2026-02-25T10:02:32.155 DEBUG:teuthology.orchestra.run.trial167:> sudo hostname trial167
2026-02-25T10:02:32.215 DEBUG:teuthology.orchestra.run.trial167:> cat /etc/os-release
2026-02-25T10:02:32.269 INFO:teuthology.orchestra.run.trial167.stdout:NAME="CentOS Stream"
2026-02-25T10:02:32.270 INFO:teuthology.orchestra.run.trial167.stdout:VERSION="9"
2026-02-25T10:02:32.270 INFO:teuthology.orchestra.run.trial167.stdout:ID="centos"
2026-02-25T10:02:32.270 INFO:teuthology.orchestra.run.trial167.stdout:ID_LIKE="rhel fedora"
2026-02-25T10:02:32.270 INFO:teuthology.orchestra.run.trial167.stdout:VERSION_ID="9"
2026-02-25T10:02:32.270 INFO:teuthology.orchestra.run.trial167.stdout:PLATFORM_ID="platform:el9"
2026-02-25T10:02:32.270 INFO:teuthology.orchestra.run.trial167.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-25T10:02:32.270 INFO:teuthology.orchestra.run.trial167.stdout:ANSI_COLOR="0;31"
2026-02-25T10:02:32.270 INFO:teuthology.orchestra.run.trial167.stdout:LOGO="fedora-logo-icon"
2026-02-25T10:02:32.271 INFO:teuthology.orchestra.run.trial167.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-25T10:02:32.271 INFO:teuthology.orchestra.run.trial167.stdout:HOME_URL="https://centos.org/"
2026-02-25T10:02:32.271 INFO:teuthology.orchestra.run.trial167.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-25T10:02:32.271 INFO:teuthology.orchestra.run.trial167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-25T10:02:32.271 INFO:teuthology.orchestra.run.trial167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-25T10:02:32.272 INFO:teuthology.provision.fog.trial167:Deploy complete!
2026-02-25T10:02:32.273 DEBUG:teuthology.parallel:result is None
2026-02-25T10:02:34.334 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-25T10:02:34.334 DEBUG:teuthology.orchestra.run.trial190:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-25T10:02:34.482 INFO:teuthology.provision.fog.trial190:Node is ready
2026-02-25T10:02:34.482 DEBUG:teuthology.orchestra.run.trial190:> hostname
2026-02-25T10:02:34.537 INFO:teuthology.orchestra.run.trial190.stdout:trial190.front.sepia.ceph.com
2026-02-25T10:02:34.537 DEBUG:teuthology.orchestra.run.trial190:> grep trial190.front.sepia.ceph.com /etc/hosts
2026-02-25T10:02:34.594 INFO:teuthology.orchestra.run.trial190.stdout:10.20.193.190 trial190.front.sepia.ceph.com trial190
2026-02-25T10:02:34.594 DEBUG:teuthology.orchestra.run.trial190:> sudo hostname trial190
2026-02-25T10:02:34.664 DEBUG:teuthology.orchestra.run.trial190:> sudo sed -i -e 's/trial190.front.sepia.ceph.com/trial190/g' /etc/hosts
2026-02-25T10:02:34.730 DEBUG:teuthology.orchestra.run.trial190:> sudo sed -i -e 's/10.20.193.190/10.20.193.190/g' /etc/hosts
2026-02-25T10:02:34.795 DEBUG:teuthology.orchestra.run.trial190:> sudo sed -i -e 's/trial190.front.sepia.ceph.com/trial190/g' /etc/hostname
2026-02-25T10:02:34.860 DEBUG:teuthology.orchestra.run.trial190:> sudo hostname trial190
2026-02-25T10:02:34.924 DEBUG:teuthology.orchestra.run.trial190:> cat /etc/os-release
2026-02-25T10:02:34.980 INFO:teuthology.orchestra.run.trial190.stdout:NAME="CentOS Stream"
2026-02-25T10:02:34.981 INFO:teuthology.orchestra.run.trial190.stdout:VERSION="9"
2026-02-25T10:02:34.981 INFO:teuthology.orchestra.run.trial190.stdout:ID="centos"
2026-02-25T10:02:34.981 INFO:teuthology.orchestra.run.trial190.stdout:ID_LIKE="rhel fedora"
2026-02-25T10:02:34.981 INFO:teuthology.orchestra.run.trial190.stdout:VERSION_ID="9"
2026-02-25T10:02:34.981 INFO:teuthology.orchestra.run.trial190.stdout:PLATFORM_ID="platform:el9"
2026-02-25T10:02:34.981 INFO:teuthology.orchestra.run.trial190.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-25T10:02:34.981 INFO:teuthology.orchestra.run.trial190.stdout:ANSI_COLOR="0;31"
2026-02-25T10:02:34.981 INFO:teuthology.orchestra.run.trial190.stdout:LOGO="fedora-logo-icon"
2026-02-25T10:02:34.982 INFO:teuthology.orchestra.run.trial190.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-25T10:02:34.982 INFO:teuthology.orchestra.run.trial190.stdout:HOME_URL="https://centos.org/"
2026-02-25T10:02:34.982 INFO:teuthology.orchestra.run.trial190.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-25T10:02:34.982 INFO:teuthology.orchestra.run.trial190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-25T10:02:34.982 INFO:teuthology.orchestra.run.trial190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-25T10:02:34.982 INFO:teuthology.provision.fog.trial190:Deploy complete!
2026-02-25T10:02:34.983 DEBUG:teuthology.parallel:result is None
2026-02-25T10:02:35.380 INFO:teuthology.lock.ops:Checking trial167.front.sepia.ceph.com
2026-02-25T10:02:35.380 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-25T10:02:35.481 INFO:teuthology.lock.ops:Checking trial190.front.sepia.ceph.com
2026-02-25T10:02:35.481 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-25T10:02:35.542 INFO:teuthology.lock.ops:Updating [trial167.front.sepia.ceph.com]: set os type and version on server
2026-02-25T10:02:35.542 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial167.front.sepia.ceph.com'
2026-02-25T10:02:35.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T10:02:35.602 DEBUG:teuthology.orchestra.run.trial167:> true
2026-02-25T10:02:35.676 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial167.front.sepia.ceph.com'
2026-02-25T10:02:35.676 DEBUG:teuthology.orchestra.run.trial167:> uname -m
2026-02-25T10:02:35.732 INFO:teuthology.orchestra.run.trial167.stdout:x86_64
2026-02-25T10:02:35.732 DEBUG:teuthology.orchestra.run.trial167:> cat /etc/os-release
2026-02-25T10:02:35.788 INFO:teuthology.orchestra.run.trial167.stdout:NAME="CentOS Stream"
2026-02-25T10:02:35.788 INFO:teuthology.orchestra.run.trial167.stdout:VERSION="9"
2026-02-25T10:02:35.788 INFO:teuthology.orchestra.run.trial167.stdout:ID="centos"
2026-02-25T10:02:35.788 INFO:teuthology.orchestra.run.trial167.stdout:ID_LIKE="rhel fedora"
2026-02-25T10:02:35.788 INFO:teuthology.orchestra.run.trial167.stdout:VERSION_ID="9"
2026-02-25T10:02:35.788 INFO:teuthology.orchestra.run.trial167.stdout:PLATFORM_ID="platform:el9"
2026-02-25T10:02:35.788 INFO:teuthology.orchestra.run.trial167.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-25T10:02:35.789 INFO:teuthology.orchestra.run.trial167.stdout:ANSI_COLOR="0;31"
2026-02-25T10:02:35.789 INFO:teuthology.orchestra.run.trial167.stdout:LOGO="fedora-logo-icon"
2026-02-25T10:02:35.789 INFO:teuthology.orchestra.run.trial167.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-25T10:02:35.789 INFO:teuthology.orchestra.run.trial167.stdout:HOME_URL="https://centos.org/"
2026-02-25T10:02:35.789 INFO:teuthology.orchestra.run.trial167.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-25T10:02:35.789 INFO:teuthology.orchestra.run.trial167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-25T10:02:35.789 INFO:teuthology.orchestra.run.trial167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-25T10:02:35.790 INFO:teuthology.lock.ops:Updating trial167.front.sepia.ceph.com on lock server
2026-02-25T10:02:35.892 INFO:teuthology.lock.ops:Updating [trial190.front.sepia.ceph.com]: set os type and version on server
2026-02-25T10:02:35.893 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial190.front.sepia.ceph.com'
2026-02-25T10:02:35.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T10:02:35.965 DEBUG:teuthology.orchestra.run.trial190:> true
2026-02-25T10:02:36.039 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial190.front.sepia.ceph.com'
2026-02-25T10:02:36.039 DEBUG:teuthology.orchestra.run.trial190:> uname -m
2026-02-25T10:02:36.096 INFO:teuthology.orchestra.run.trial190.stdout:x86_64
2026-02-25T10:02:36.096 DEBUG:teuthology.orchestra.run.trial190:> cat /etc/os-release
2026-02-25T10:02:36.153 INFO:teuthology.orchestra.run.trial190.stdout:NAME="CentOS Stream"
2026-02-25T10:02:36.153 INFO:teuthology.orchestra.run.trial190.stdout:VERSION="9"
2026-02-25T10:02:36.153 INFO:teuthology.orchestra.run.trial190.stdout:ID="centos"
2026-02-25T10:02:36.165 INFO:teuthology.orchestra.run.trial190.stdout:ID_LIKE="rhel fedora"
2026-02-25T10:02:36.165 INFO:teuthology.orchestra.run.trial190.stdout:VERSION_ID="9"
2026-02-25T10:02:36.165 INFO:teuthology.orchestra.run.trial190.stdout:PLATFORM_ID="platform:el9"
2026-02-25T10:02:36.165 INFO:teuthology.orchestra.run.trial190.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-25T10:02:36.165 INFO:teuthology.orchestra.run.trial190.stdout:ANSI_COLOR="0;31"
2026-02-25T10:02:36.165 INFO:teuthology.orchestra.run.trial190.stdout:LOGO="fedora-logo-icon"
2026-02-25T10:02:36.165 INFO:teuthology.orchestra.run.trial190.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-25T10:02:36.165 INFO:teuthology.orchestra.run.trial190.stdout:HOME_URL="https://centos.org/"
2026-02-25T10:02:36.165 INFO:teuthology.orchestra.run.trial190.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-25T10:02:36.166 INFO:teuthology.orchestra.run.trial190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-25T10:02:36.166 INFO:teuthology.orchestra.run.trial190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-25T10:02:36.166 INFO:teuthology.lock.ops:Updating trial190.front.sepia.ceph.com on lock server
2026-02-25T10:02:36.270 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:02:36.362 INFO:teuthology.dispatcher.supervisor:Running job 69392
2026-02-25T10:02:36.362 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/bin/teuthology -v --owner scheduled_matan@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/matan-2026-02-25_08:02:37-crimson-rados-main-distro-default-trial/69392 --name matan-2026-02-25_08:02:37-crimson-rados-main-distro-default-trial --description crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/crimson_bluestore thrashers/{pggrow} thrashosds-health workloads/radosbench-high-concurrency} -- /home/teuthworker/mnt/teuthology/matan-2026-02-25_08:02:37-crimson-rados-main-distro-default-trial/69392/orig.config.yaml
2026-02-25T10:02:36.363 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/matan-2026-02-25_08:02:37-crimson-rados-main-distro-default-trial/69392
2026-02-25T10:02:36.363 INFO:teuthology.dispatcher.supervisor:Job PID: 4165160
2026-02-25T10:02:36.363 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-25T10:04:36.363 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:06:36.485 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:08:36.609 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:10:36.731 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:12:36.843 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:14:36.965 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:16:37.090 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:18:37.177 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:20:37.296 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:22:37.427 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:24:37.554 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:26:37.675 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:28:37.804 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:30:37.932 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:32:38.053 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:34:38.176 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:36:38.271 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:38:38.402 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:40:38.527 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:42:38.649 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:44:38.768 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:46:38.881 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:48:38.998 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:50:39.193 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:52:39.271 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:54:39.394 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:56:39.485 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T10:58:39.837 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:00:40.051 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:02:40.178 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:04:41.320 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:06:41.449 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:08:41.576 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T11:08:41.697 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
2026-02-25T11:08:41.819 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-25T11:08:42.327 INFO:teuthology.orchestra.console.trial167:Power off
2026-02-25T11:08:42.327 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-25T11:08:42.347 INFO:teuthology.orchestra.console.trial190:Power off
2026-02-25T11:08:42.347 DEBUG:teuthology.orchestra.console.trial190:pexpect command: ipmitool -H trial190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-25T11:08:42.366 DEBUG:teuthology.orchestra.console.trial167:power off output: Chassis Power Control: Down/Off
2026-02-25T11:08:42.385 DEBUG:teuthology.orchestra.console.trial190:power off output: Chassis Power Control: Down/Off
2026-02-25T11:08:46.367 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T11:08:46.385 DEBUG:teuthology.orchestra.console.trial190:pexpect command: ipmitool -H trial190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T11:08:46.478 DEBUG:teuthology.orchestra.console.trial167:check power output: Chassis Power is on
2026-02-25T11:08:46.494 DEBUG:teuthology.orchestra.console.trial190:check power output: Chassis Power is on
2026-02-25T11:08:50.479 DEBUG:teuthology.orchestra.console.trial167:pexpect command: ipmitool -H trial167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T11:08:50.495 DEBUG:teuthology.orchestra.console.trial190:pexpect command: ipmitool -H trial190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T11:08:50.589 DEBUG:teuthology.orchestra.console.trial167:check power output: Chassis Power is off
2026-02-25T11:08:50.589 INFO:teuthology.orchestra.console.trial167:Power off completed
2026-02-25T11:08:50.605 DEBUG:teuthology.orchestra.console.trial190:check power output: Chassis Power is off
2026-02-25T11:08:50.605 INFO:teuthology.orchestra.console.trial190:Power off completed
2026-02-25T11:08:50.837 INFO:teuthology.lock.ops:unlocked: trial167.front.sepia.ceph.com
2026-02-25T11:08:50.849 INFO:teuthology.lock.ops:unlocked: trial190.front.sepia.ceph.com