2026-02-13T18:54:52.577 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71
2026-02-13T18:54:52.577 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T18:54:54.820 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T18:54:55.014 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x72d1a583d970>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x72d1a502a020>, signals=[15])
2026-02-13T18:54:55.014 INFO:teuthology.lock.ops:Start node 'trial043.front.sepia.ceph.com' reimaging
2026-02-13T18:54:55.014 INFO:teuthology.lock.ops:Updating [trial043.front.sepia.ceph.com]: reset os type and version on server
2026-02-13T18:54:55.014 INFO:teuthology.lock.ops:Updating trial043.front.sepia.ceph.com on lock server
2026-02-13T18:54:55.069 INFO:teuthology.lock.ops:Start node 'trial125.front.sepia.ceph.com' reimaging
2026-02-13T18:54:55.069 INFO:teuthology.lock.ops:Updating [trial125.front.sepia.ceph.com]: reset os type and version on server
2026-02-13T18:54:55.069 INFO:teuthology.lock.ops:Updating trial125.front.sepia.ceph.com on lock server
2026-02-13T18:54:55.284 DEBUG:teuthology.provision.fog.trial125:Requesting image trial_centos_9.stream (ID 3)
2026-02-13T18:54:55.299 DEBUG:teuthology.provision.fog.trial043:Requesting image trial_centos_9.stream (ID 3)
2026-02-13T18:54:55.326 INFO:teuthology.provision.fog.trial043:Scheduling deploy of centos 9.stream
2026-02-13T18:54:55.335 INFO:teuthology.provision.fog.trial125:Scheduling deploy of centos 9.stream
2026-02-13T18:54:56.515 INFO:teuthology.orchestra.console.trial043:Power off
2026-02-13T18:54:56.515 DEBUG:teuthology.orchestra.console.trial043:pexpect command: ipmitool -H trial043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-13T18:54:56.536 INFO:teuthology.orchestra.console.trial125:Power off
2026-02-13T18:54:56.536 DEBUG:teuthology.orchestra.console.trial125:pexpect command: ipmitool -H trial125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-13T18:54:56.554 DEBUG:teuthology.orchestra.console.trial043:power off output: Chassis Power Control: Down/Off
2026-02-13T18:54:56.572 DEBUG:teuthology.orchestra.console.trial125:power off output: Chassis Power Control: Down/Off
2026-02-13T18:55:00.559 DEBUG:teuthology.orchestra.console.trial043:pexpect command: ipmitool -H trial043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T18:55:00.572 DEBUG:teuthology.orchestra.console.trial125:pexpect command: ipmitool -H trial125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T18:55:00.668 DEBUG:teuthology.orchestra.console.trial043:check power output: Chassis Power is off
2026-02-13T18:55:00.669 INFO:teuthology.orchestra.console.trial043:Power off completed
2026-02-13T18:55:00.680 DEBUG:teuthology.orchestra.console.trial125:check power output: Chassis Power is off
2026-02-13T18:55:00.681 INFO:teuthology.orchestra.console.trial125:Power off completed
2026-02-13T18:55:00.770 INFO:teuthology.orchestra.console.trial043:Power on
2026-02-13T18:55:00.770 DEBUG:teuthology.orchestra.console.trial043:pexpect command: ipmitool -H trial043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-13T18:55:00.782 INFO:teuthology.orchestra.console.trial125:Power on
2026-02-13T18:55:00.782 DEBUG:teuthology.orchestra.console.trial125:pexpect command: ipmitool -H trial125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-13T18:55:00.809 DEBUG:teuthology.orchestra.console.trial043:power on output: Chassis Power Control: Up/On
2026-02-13T18:55:00.821 DEBUG:teuthology.orchestra.console.trial125:power on output: Chassis Power Control: Up/On
2026-02-13T18:55:04.813 DEBUG:teuthology.orchestra.console.trial043:pexpect command: ipmitool -H trial043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T18:55:04.823 DEBUG:teuthology.orchestra.console.trial125:pexpect command: ipmitool -H trial125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T18:55:04.925 DEBUG:teuthology.orchestra.console.trial043:check power output: Chassis Power is off
2026-02-13T18:55:04.931 DEBUG:teuthology.orchestra.console.trial125:check power output: Chassis Power is off
2026-02-13T18:55:08.930 DEBUG:teuthology.orchestra.console.trial043:pexpect command: ipmitool -H trial043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T18:55:08.938 DEBUG:teuthology.orchestra.console.trial125:pexpect command: ipmitool -H trial125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T18:55:09.040 DEBUG:teuthology.orchestra.console.trial043:check power output: Chassis Power is on
2026-02-13T18:55:09.041 INFO:teuthology.orchestra.console.trial043:Power on completed
2026-02-13T18:55:09.048 DEBUG:teuthology.orchestra.console.trial125:check power output: Chassis Power is on
2026-02-13T18:55:09.048 INFO:teuthology.orchestra.console.trial125:Power on completed
2026-02-13T18:55:09.142 INFO:teuthology.provision.fog.trial043:Waiting for deploy to finish
2026-02-13T18:55:09.149 INFO:teuthology.provision.fog.trial125:Waiting for deploy to finish
2026-02-13T18:56:56.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T18:56:56.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T18:57:10.813 WARNING:teuthology.provision.fog.trial043:[Errno None] Unable to connect to port 22 on 10.20.193.43
2026-02-13T18:57:10.813 WARNING:teuthology.provision.fog.trial125:[Errno None] Unable to connect to port 22 on 10.20.193.125
2026-02-13T18:57:16.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T18:57:16.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T18:57:19.901 WARNING:teuthology.provision.fog.trial043:[Errno None] Unable to connect to port 22 on 10.20.193.43
2026-02-13T18:57:19.901 WARNING:teuthology.provision.fog.trial125:[Errno None] Unable to connect to port 22 on 10.20.193.125
2026-02-13T18:57:25.908 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T18:57:25.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T18:57:28.989 WARNING:teuthology.provision.fog.trial125:[Errno None] Unable to connect to port 22 on 10.20.193.125
2026-02-13T18:57:28.989 WARNING:teuthology.provision.fog.trial043:[Errno None] Unable to connect to port 22 on 10.20.193.43
2026-02-13T18:57:34.996 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T18:57:34.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T18:57:38.077 WARNING:teuthology.provision.fog.trial125:[Errno None] Unable to connect to port 22 on 10.20.193.125
2026-02-13T18:57:44.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T18:58:10.853 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-13T18:58:10.853 DEBUG:teuthology.orchestra.run.trial043:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-13T18:58:16.018 INFO:teuthology.provision.fog.trial043:Node is ready
2026-02-13T18:58:16.018 DEBUG:teuthology.orchestra.run.trial043:> hostname
2026-02-13T18:58:16.075 INFO:teuthology.orchestra.run.trial043.stdout:trial043.front.sepia.ceph.com
2026-02-13T18:58:16.075 DEBUG:teuthology.orchestra.run.trial043:> grep trial043.front.sepia.ceph.com /etc/hosts
2026-02-13T18:58:16.131 INFO:teuthology.orchestra.run.trial043.stdout:10.20.193.43 trial043.front.sepia.ceph.com trial043
2026-02-13T18:58:16.131 DEBUG:teuthology.orchestra.run.trial043:> sudo hostname trial043
2026-02-13T18:58:16.201 DEBUG:teuthology.orchestra.run.trial043:> sudo sed -i -e 's/trial043.front.sepia.ceph.com/trial043/g' /etc/hosts
2026-02-13T18:58:16.266 DEBUG:teuthology.orchestra.run.trial043:> sudo sed -i -e 's/10.20.193.43/10.20.193.43/g' /etc/hosts
2026-02-13T18:58:16.328 DEBUG:teuthology.orchestra.run.trial043:> sudo sed -i -e 's/trial043.front.sepia.ceph.com/trial043/g' /etc/hostname
2026-02-13T18:58:16.393 DEBUG:teuthology.orchestra.run.trial043:> sudo hostname trial043
2026-02-13T18:58:16.457 DEBUG:teuthology.orchestra.run.trial043:> cat /etc/os-release
2026-02-13T18:58:16.514 INFO:teuthology.orchestra.run.trial043.stdout:NAME="CentOS Stream"
2026-02-13T18:58:16.514 INFO:teuthology.orchestra.run.trial043.stdout:VERSION="9"
2026-02-13T18:58:16.514 INFO:teuthology.orchestra.run.trial043.stdout:ID="centos"
2026-02-13T18:58:16.514 INFO:teuthology.orchestra.run.trial043.stdout:ID_LIKE="rhel fedora"
2026-02-13T18:58:16.514 INFO:teuthology.orchestra.run.trial043.stdout:VERSION_ID="9"
2026-02-13T18:58:16.514 INFO:teuthology.orchestra.run.trial043.stdout:PLATFORM_ID="platform:el9"
2026-02-13T18:58:16.515 INFO:teuthology.orchestra.run.trial043.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-13T18:58:16.515 INFO:teuthology.orchestra.run.trial043.stdout:ANSI_COLOR="0;31"
2026-02-13T18:58:16.515 INFO:teuthology.orchestra.run.trial043.stdout:LOGO="fedora-logo-icon"
2026-02-13T18:58:16.515 INFO:teuthology.orchestra.run.trial043.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-13T18:58:16.515 INFO:teuthology.orchestra.run.trial043.stdout:HOME_URL="https://centos.org/"
2026-02-13T18:58:16.515 INFO:teuthology.orchestra.run.trial043.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-13T18:58:16.515 INFO:teuthology.orchestra.run.trial043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-13T18:58:16.515 INFO:teuthology.orchestra.run.trial043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-13T18:58:16.516 INFO:teuthology.provision.fog.trial043:Deploy complete!
2026-02-13T18:58:16.517 DEBUG:teuthology.parallel:result is None
2026-02-13T18:58:20.022 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-13T18:58:20.022 DEBUG:teuthology.orchestra.run.trial125:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-13T18:58:20.174 INFO:teuthology.provision.fog.trial125:Node is ready
2026-02-13T18:58:20.174 DEBUG:teuthology.orchestra.run.trial125:> hostname
2026-02-13T18:58:20.229 INFO:teuthology.orchestra.run.trial125.stdout:trial125.front.sepia.ceph.com
2026-02-13T18:58:20.230 DEBUG:teuthology.orchestra.run.trial125:> grep trial125.front.sepia.ceph.com /etc/hosts
2026-02-13T18:58:20.285 INFO:teuthology.orchestra.run.trial125.stdout:10.20.193.125 trial125.front.sepia.ceph.com trial125
2026-02-13T18:58:20.286 DEBUG:teuthology.orchestra.run.trial125:> sudo hostname trial125
2026-02-13T18:58:20.355 DEBUG:teuthology.orchestra.run.trial125:> sudo sed -i -e 's/trial125.front.sepia.ceph.com/trial125/g' /etc/hosts
2026-02-13T18:58:20.419 DEBUG:teuthology.orchestra.run.trial125:> sudo sed -i -e 's/10.20.193.125/10.20.193.125/g' /etc/hosts
2026-02-13T18:58:20.483 DEBUG:teuthology.orchestra.run.trial125:> sudo sed -i -e 's/trial125.front.sepia.ceph.com/trial125/g' /etc/hostname
2026-02-13T18:58:20.547 DEBUG:teuthology.orchestra.run.trial125:> sudo hostname trial125
2026-02-13T18:58:20.611 DEBUG:teuthology.orchestra.run.trial125:> cat /etc/os-release
2026-02-13T18:58:20.667 INFO:teuthology.orchestra.run.trial125.stdout:NAME="CentOS Stream"
2026-02-13T18:58:20.667 INFO:teuthology.orchestra.run.trial125.stdout:VERSION="9"
2026-02-13T18:58:20.667 INFO:teuthology.orchestra.run.trial125.stdout:ID="centos"
2026-02-13T18:58:20.668 INFO:teuthology.orchestra.run.trial125.stdout:ID_LIKE="rhel fedora"
2026-02-13T18:58:20.668 INFO:teuthology.orchestra.run.trial125.stdout:VERSION_ID="9"
2026-02-13T18:58:20.668 INFO:teuthology.orchestra.run.trial125.stdout:PLATFORM_ID="platform:el9"
2026-02-13T18:58:20.668 INFO:teuthology.orchestra.run.trial125.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-13T18:58:20.668 INFO:teuthology.orchestra.run.trial125.stdout:ANSI_COLOR="0;31"
2026-02-13T18:58:20.668 INFO:teuthology.orchestra.run.trial125.stdout:LOGO="fedora-logo-icon"
2026-02-13T18:58:20.668 INFO:teuthology.orchestra.run.trial125.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-13T18:58:20.668 INFO:teuthology.orchestra.run.trial125.stdout:HOME_URL="https://centos.org/"
2026-02-13T18:58:20.668 INFO:teuthology.orchestra.run.trial125.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-13T18:58:20.668 INFO:teuthology.orchestra.run.trial125.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-13T18:58:20.668 INFO:teuthology.orchestra.run.trial125.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-13T18:58:20.669 INFO:teuthology.provision.fog.trial125:Deploy complete!
2026-02-13T18:58:20.670 DEBUG:teuthology.parallel:result is None
2026-02-13T18:58:21.077 INFO:teuthology.lock.ops:Checking trial043.front.sepia.ceph.com
2026-02-13T18:58:21.077 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-13T18:58:21.163 INFO:teuthology.lock.ops:Checking trial125.front.sepia.ceph.com
2026-02-13T18:58:21.163 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-13T18:58:21.224 INFO:teuthology.lock.ops:Updating [trial043.front.sepia.ceph.com]: set os type and version on server
2026-02-13T18:58:21.225 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial043.front.sepia.ceph.com'
2026-02-13T18:58:21.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T18:58:21.287 DEBUG:teuthology.orchestra.run.trial043:> true
2026-02-13T18:58:21.357 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial043.front.sepia.ceph.com'
2026-02-13T18:58:21.357 DEBUG:teuthology.orchestra.run.trial043:> uname -m
2026-02-13T18:58:21.413 INFO:teuthology.orchestra.run.trial043.stdout:x86_64
2026-02-13T18:58:21.413 DEBUG:teuthology.orchestra.run.trial043:> cat /etc/os-release
2026-02-13T18:58:21.469 INFO:teuthology.orchestra.run.trial043.stdout:NAME="CentOS Stream"
2026-02-13T18:58:21.469 INFO:teuthology.orchestra.run.trial043.stdout:VERSION="9"
2026-02-13T18:58:21.469 INFO:teuthology.orchestra.run.trial043.stdout:ID="centos"
2026-02-13T18:58:21.469 INFO:teuthology.orchestra.run.trial043.stdout:ID_LIKE="rhel fedora"
2026-02-13T18:58:21.469 INFO:teuthology.orchestra.run.trial043.stdout:VERSION_ID="9"
2026-02-13T18:58:21.470 INFO:teuthology.orchestra.run.trial043.stdout:PLATFORM_ID="platform:el9"
2026-02-13T18:58:21.470 INFO:teuthology.orchestra.run.trial043.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-13T18:58:21.470 INFO:teuthology.orchestra.run.trial043.stdout:ANSI_COLOR="0;31"
2026-02-13T18:58:21.470 INFO:teuthology.orchestra.run.trial043.stdout:LOGO="fedora-logo-icon"
2026-02-13T18:58:21.470 INFO:teuthology.orchestra.run.trial043.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-13T18:58:21.470 INFO:teuthology.orchestra.run.trial043.stdout:HOME_URL="https://centos.org/"
2026-02-13T18:58:21.470 INFO:teuthology.orchestra.run.trial043.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-13T18:58:21.470 INFO:teuthology.orchestra.run.trial043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-13T18:58:21.470 INFO:teuthology.orchestra.run.trial043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-13T18:58:21.471 INFO:teuthology.lock.ops:Updating trial043.front.sepia.ceph.com on lock server
2026-02-13T18:58:21.573 INFO:teuthology.lock.ops:Updating [trial125.front.sepia.ceph.com]: set os type and version on server
2026-02-13T18:58:21.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial125.front.sepia.ceph.com'
2026-02-13T18:58:21.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-13T18:58:21.645 DEBUG:teuthology.orchestra.run.trial125:> true
2026-02-13T18:58:21.720 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial125.front.sepia.ceph.com'
2026-02-13T18:58:21.720 DEBUG:teuthology.orchestra.run.trial125:> uname -m
2026-02-13T18:58:21.776 INFO:teuthology.orchestra.run.trial125.stdout:x86_64
2026-02-13T18:58:21.776 DEBUG:teuthology.orchestra.run.trial125:> cat /etc/os-release
2026-02-13T18:58:21.832 INFO:teuthology.orchestra.run.trial125.stdout:NAME="CentOS Stream"
2026-02-13T18:58:21.832 INFO:teuthology.orchestra.run.trial125.stdout:VERSION="9"
2026-02-13T18:58:21.832 INFO:teuthology.orchestra.run.trial125.stdout:ID="centos"
2026-02-13T18:58:21.832 INFO:teuthology.orchestra.run.trial125.stdout:ID_LIKE="rhel fedora"
2026-02-13T18:58:21.832 INFO:teuthology.orchestra.run.trial125.stdout:VERSION_ID="9"
2026-02-13T18:58:21.832 INFO:teuthology.orchestra.run.trial125.stdout:PLATFORM_ID="platform:el9"
2026-02-13T18:58:21.832 INFO:teuthology.orchestra.run.trial125.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-13T18:58:21.833 INFO:teuthology.orchestra.run.trial125.stdout:ANSI_COLOR="0;31"
2026-02-13T18:58:21.833 INFO:teuthology.orchestra.run.trial125.stdout:LOGO="fedora-logo-icon"
2026-02-13T18:58:21.833 INFO:teuthology.orchestra.run.trial125.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-13T18:58:21.833 INFO:teuthology.orchestra.run.trial125.stdout:HOME_URL="https://centos.org/"
2026-02-13T18:58:21.833 INFO:teuthology.orchestra.run.trial125.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-13T18:58:21.833 INFO:teuthology.orchestra.run.trial125.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-13T18:58:21.833 INFO:teuthology.orchestra.run.trial125.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-13T18:58:21.834 INFO:teuthology.lock.ops:Updating trial125.front.sepia.ceph.com on lock server
2026-02-13T18:58:21.938 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T18:58:22.032 INFO:teuthology.dispatcher.supervisor:Running job 49767
2026-02-13T18:58:22.032 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/bin/teuthology -v --owner scheduled_yuriw@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/yuriw-2026-02-13_16:37:03-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/49767 --name yuriw-2026-02-13_16:37:03-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial --description rados/mgr/{clusters/{2-node-mgr} debug/mgr distro/{centos_latest} mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore/{alloc$/{bitmap} base mem$/{low} onode-segment$/{256K} write$/{v2/{compr$/{yes$/{zstd}} v2}}}} tasks/{1-install 2-ceph 3-mgrmodules 4-units/failover}} -- /home/teuthworker/mnt/teuthology/yuriw-2026-02-13_16:37:03-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/49767/orig.config.yaml
2026-02-13T18:58:22.033 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/yuriw-2026-02-13_16:37:03-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/49767
2026-02-13T18:58:22.033 INFO:teuthology.dispatcher.supervisor:Job PID: 3333048
2026-02-13T18:58:22.033 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-13T19:00:22.108 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T19:02:22.342 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T19:04:22.533 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T19:06:22.672 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T19:08:22.833 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T19:10:22.997 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-13T19:10:23.080 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-13T19:10:23.194 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-13T19:10:23.712 INFO:teuthology.orchestra.console.trial125:Power off
2026-02-13T19:10:23.713 DEBUG:teuthology.orchestra.console.trial125:pexpect command: ipmitool -H trial125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-13T19:10:23.722 INFO:teuthology.orchestra.console.trial043:Power off
2026-02-13T19:10:23.723 DEBUG:teuthology.orchestra.console.trial043:pexpect command: ipmitool -H trial043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-13T19:10:23.751 DEBUG:teuthology.orchestra.console.trial125:power off output: Chassis Power Control: Down/Off
2026-02-13T19:10:23.761 DEBUG:teuthology.orchestra.console.trial043:power off output: Chassis Power Control: Down/Off
2026-02-13T19:10:27.756 DEBUG:teuthology.orchestra.console.trial125:pexpect command: ipmitool -H trial125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T19:10:27.763 DEBUG:teuthology.orchestra.console.trial043:pexpect command: ipmitool -H trial043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T19:10:27.865 DEBUG:teuthology.orchestra.console.trial125:check power output: Chassis Power is on
2026-02-13T19:10:27.873 DEBUG:teuthology.orchestra.console.trial043:check power output: Chassis Power is on
2026-02-13T19:10:31.870 DEBUG:teuthology.orchestra.console.trial125:pexpect command: ipmitool -H trial125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T19:10:31.877 DEBUG:teuthology.orchestra.console.trial043:pexpect command: ipmitool -H trial043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-13T19:10:31.979 DEBUG:teuthology.orchestra.console.trial125:check power output: Chassis Power is off
2026-02-13T19:10:31.979 INFO:teuthology.orchestra.console.trial125:Power off completed
2026-02-13T19:10:31.987 DEBUG:teuthology.orchestra.console.trial043:check power output: Chassis Power is off
2026-02-13T19:10:31.987 INFO:teuthology.orchestra.console.trial043:Power off completed
2026-02-13T19:10:32.239 INFO:teuthology.lock.ops:unlocked: trial043.front.sepia.ceph.com
2026-02-13T19:11:49.423 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/bin/teuthology-supervisor", line 6, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/scripts/supervisor.py", line 40, in main
    sys.exit(teuthology.dispatcher.supervisor.main(parse_args(sys.argv[1:])))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/dispatcher/supervisor.py", line 71, in main
    return run_job(
           ^^^^^^^^
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/dispatcher/supervisor.py", line 179, in run_job
    unlock_targets(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/dispatcher/supervisor.py", line 279, in unlock_targets
    lock_ops.unlock_safe(locked, job_config["owner"], job_config["name"], job_config["job_id"])
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/lock/ops.py", line 188, in unlock_safe
    return all(p)
           ^^^^^^
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/lock/ops.py", line 198, in unlock_one_safe
    return unlock_one(name, owner, node_status["description"], node_status)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/lock/ops.py", line 256, in unlock_one
    while proceed():
          ^^^^^^^^^
  File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/contextutil.py", line 134, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'unlock trial125.front.sepia.ceph.com' reached maximum tries (10) after waiting for 32.5 seconds