2026-02-25T15:08:58.804 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-25T15:08:58.804 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:08:58.882 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:08:59.196 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x73503f8a1f10>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x73503f089080>, signals=[15])
2026-02-25T15:08:59.196 INFO:teuthology.lock.ops:Start node 'trial050.front.sepia.ceph.com' reimaging
2026-02-25T15:08:59.196 INFO:teuthology.lock.ops:Updating [trial050.front.sepia.ceph.com]: reset os type and version on server
2026-02-25T15:08:59.196 INFO:teuthology.lock.ops:Updating trial050.front.sepia.ceph.com on lock server
2026-02-25T15:08:59.256 INFO:teuthology.lock.ops:Start node 'trial145.front.sepia.ceph.com' reimaging
2026-02-25T15:08:59.256 INFO:teuthology.lock.ops:Updating [trial145.front.sepia.ceph.com]: reset os type and version on server
2026-02-25T15:08:59.256 INFO:teuthology.lock.ops:Updating trial145.front.sepia.ceph.com on lock server
2026-02-25T15:08:59.455 DEBUG:teuthology.provision.fog.trial145:Requesting image trial_ubuntu_22.04 (ID 1)
2026-02-25T15:08:59.485 DEBUG:teuthology.provision.fog.trial050:Requesting image trial_ubuntu_22.04 (ID 1)
2026-02-25T15:08:59.501 INFO:teuthology.provision.fog.trial145:Scheduling deploy of ubuntu 22.04
2026-02-25T15:08:59.506 INFO:teuthology.provision.fog.trial050:Scheduling deploy of ubuntu 22.04
2026-02-25T15:09:00.669 INFO:teuthology.orchestra.console.trial050:Power off
2026-02-25T15:09:00.669 DEBUG:teuthology.orchestra.console.trial050:pexpect command: ipmitool -H trial050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-25T15:09:00.689 INFO:teuthology.orchestra.console.trial145:Power off
2026-02-25T15:09:00.689 DEBUG:teuthology.orchestra.console.trial145:pexpect command: ipmitool -H trial145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-25T15:09:00.708 DEBUG:teuthology.orchestra.console.trial050:power off output: Chassis Power Control: Down/Off
2026-02-25T15:09:00.726 DEBUG:teuthology.orchestra.console.trial145:power off output: Chassis Power Control: Down/Off
2026-02-25T15:09:04.712 DEBUG:teuthology.orchestra.console.trial050:pexpect command: ipmitool -H trial050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T15:09:04.727 DEBUG:teuthology.orchestra.console.trial145:pexpect command: ipmitool -H trial145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T15:09:04.822 DEBUG:teuthology.orchestra.console.trial050:check power output: Chassis Power is off
2026-02-25T15:09:04.823 INFO:teuthology.orchestra.console.trial050:Power off completed
2026-02-25T15:09:04.835 DEBUG:teuthology.orchestra.console.trial145:check power output: Chassis Power is off
2026-02-25T15:09:04.835 INFO:teuthology.orchestra.console.trial145:Power off completed
2026-02-25T15:09:04.923 INFO:teuthology.orchestra.console.trial050:Power on
2026-02-25T15:09:04.923 DEBUG:teuthology.orchestra.console.trial050:pexpect command: ipmitool -H trial050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-25T15:09:04.936 INFO:teuthology.orchestra.console.trial145:Power on
2026-02-25T15:09:04.936 DEBUG:teuthology.orchestra.console.trial145:pexpect command: ipmitool -H trial145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-25T15:09:04.962 DEBUG:teuthology.orchestra.console.trial050:power on output: Chassis Power Control: Up/On
2026-02-25T15:09:04.973 DEBUG:teuthology.orchestra.console.trial145:power on output: Chassis Power Control: Up/On
2026-02-25T15:09:08.966 DEBUG:teuthology.orchestra.console.trial050:pexpect command: ipmitool -H trial050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T15:09:08.975 DEBUG:teuthology.orchestra.console.trial145:pexpect command: ipmitool -H trial145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T15:09:09.077 DEBUG:teuthology.orchestra.console.trial050:check power output: Chassis Power is on
2026-02-25T15:09:09.077 INFO:teuthology.orchestra.console.trial050:Power on completed
2026-02-25T15:09:09.085 DEBUG:teuthology.orchestra.console.trial145:check power output: Chassis Power is on
2026-02-25T15:09:09.085 INFO:teuthology.orchestra.console.trial145:Power on completed
2026-02-25T15:09:09.178 INFO:teuthology.provision.fog.trial050:Waiting for deploy to finish
2026-02-25T15:09:09.186 INFO:teuthology.provision.fog.trial145:Waiting for deploy to finish
2026-02-25T15:11:12.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T15:11:12.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T15:11:15.328 WARNING:teuthology.provision.fog.trial145:[Errno None] Unable to connect to port 22 on 10.20.193.145
2026-02-25T15:11:15.392 WARNING:teuthology.provision.fog.trial050:[Errno None] Unable to connect to port 22 on 10.20.193.50
2026-02-25T15:11:21.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T15:11:21.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T15:11:24.416 WARNING:teuthology.provision.fog.trial145:[Errno None] Unable to connect to port 22 on 10.20.193.145
2026-02-25T15:11:24.480 WARNING:teuthology.provision.fog.trial050:[Errno None] Unable to connect to port 22 on 10.20.193.50
2026-02-25T15:11:30.424 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T15:11:30.481 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T15:11:33.504 WARNING:teuthology.provision.fog.trial145:[Errno None] Unable to connect to port 22 on 10.20.193.145
2026-02-25T15:11:33.568 WARNING:teuthology.provision.fog.trial050:[Errno None] Unable to connect to port 22 on 10.20.193.50
2026-02-25T15:11:39.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T15:11:39.569 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T15:11:42.592 WARNING:teuthology.provision.fog.trial145:[Errno None] Unable to connect to port 22 on 10.20.193.145
2026-02-25T15:11:42.656 WARNING:teuthology.provision.fog.trial050:[Errno None] Unable to connect to port 22 on 10.20.193.50
2026-02-25T15:11:48.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T15:11:48.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T15:12:48.662 WARNING:teuthology.provision.fog.trial145:timed out
2026-02-25T15:12:48.662 WARNING:teuthology.provision.fog.trial050:timed out
2026-02-25T15:12:54.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T15:12:54.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T15:12:54.827 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-25T15:12:54.827 DEBUG:teuthology.orchestra.run.trial050:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-25T15:12:54.829 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-25T15:12:54.829 DEBUG:teuthology.orchestra.run.trial145:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-25T15:12:55.222 INFO:teuthology.provision.fog.trial050:Node is ready
2026-02-25T15:12:55.222 DEBUG:teuthology.orchestra.run.trial050:> hostname
2026-02-25T15:12:55.241 INFO:teuthology.provision.fog.trial145:Node is ready
2026-02-25T15:12:55.241 DEBUG:teuthology.orchestra.run.trial145:> hostname
2026-02-25T15:12:55.267 INFO:teuthology.orchestra.run.trial050.stdout:trial050.front.sepia.ceph.com
2026-02-25T15:12:55.267 DEBUG:teuthology.orchestra.run.trial050:> grep trial050.front.sepia.ceph.com /etc/hosts
2026-02-25T15:12:55.291 INFO:teuthology.orchestra.run.trial145.stdout:trial145.front.sepia.ceph.com
2026-02-25T15:12:55.292 DEBUG:teuthology.orchestra.run.trial145:> grep trial145.front.sepia.ceph.com /etc/hosts
2026-02-25T15:12:55.311 INFO:teuthology.orchestra.run.trial050.stdout:10.20.193.50 trial050.front.sepia.ceph.com trial050
2026-02-25T15:12:55.311 DEBUG:teuthology.orchestra.run.trial050:> sudo hostname trial050
2026-02-25T15:12:55.335 INFO:teuthology.orchestra.run.trial145.stdout:10.20.193.145 trial145.front.sepia.ceph.com trial145
2026-02-25T15:12:55.336 DEBUG:teuthology.orchestra.run.trial145:> sudo hostname trial145
2026-02-25T15:12:55.362 DEBUG:teuthology.orchestra.run.trial050:> sudo sed -i -e 's/trial050.front.sepia.ceph.com/trial050/g' /etc/hosts
2026-02-25T15:12:55.387 DEBUG:teuthology.orchestra.run.trial145:> sudo sed -i -e 's/trial145.front.sepia.ceph.com/trial145/g' /etc/hosts
2026-02-25T15:12:55.411 DEBUG:teuthology.orchestra.run.trial050:> sudo sed -i -e 's/10.20.193.50/10.20.193.50/g' /etc/hosts
2026-02-25T15:12:55.435 DEBUG:teuthology.orchestra.run.trial145:> sudo sed -i -e 's/10.20.193.145/10.20.193.145/g' /etc/hosts
2026-02-25T15:12:55.463 DEBUG:teuthology.orchestra.run.trial050:> sudo sed -i -e 's/trial050.front.sepia.ceph.com/trial050/g' /etc/hostname
2026-02-25T15:12:55.483 DEBUG:teuthology.orchestra.run.trial145:> sudo sed -i -e 's/trial145.front.sepia.ceph.com/trial145/g' /etc/hostname
2026-02-25T15:12:55.511 DEBUG:teuthology.orchestra.run.trial050:> sudo hostname trial050
2026-02-25T15:12:55.536 DEBUG:teuthology.orchestra.run.trial145:> sudo hostname trial145
2026-02-25T15:12:55.567 DEBUG:teuthology.orchestra.run.trial050:> cat /etc/os-release
2026-02-25T15:12:55.584 DEBUG:teuthology.orchestra.run.trial145:> cat /etc/os-release
2026-02-25T15:12:55.611 INFO:teuthology.orchestra.run.trial050.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS"
2026-02-25T15:12:55.611 INFO:teuthology.orchestra.run.trial050.stdout:NAME="Ubuntu"
2026-02-25T15:12:55.611 INFO:teuthology.orchestra.run.trial050.stdout:VERSION_ID="22.04"
2026-02-25T15:12:55.611 INFO:teuthology.orchestra.run.trial050.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)"
2026-02-25T15:12:55.611 INFO:teuthology.orchestra.run.trial050.stdout:VERSION_CODENAME=jammy
2026-02-25T15:12:55.611 INFO:teuthology.orchestra.run.trial050.stdout:ID=ubuntu
2026-02-25T15:12:55.611 INFO:teuthology.orchestra.run.trial050.stdout:ID_LIKE=debian
2026-02-25T15:12:55.611 INFO:teuthology.orchestra.run.trial050.stdout:HOME_URL="https://www.ubuntu.com/"
2026-02-25T15:12:55.612 INFO:teuthology.orchestra.run.trial050.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2026-02-25T15:12:55.612 INFO:teuthology.orchestra.run.trial050.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2026-02-25T15:12:55.612 INFO:teuthology.orchestra.run.trial050.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2026-02-25T15:12:55.612 INFO:teuthology.orchestra.run.trial050.stdout:UBUNTU_CODENAME=jammy
2026-02-25T15:12:55.613 INFO:teuthology.provision.fog.trial050:Deploy complete!
2026-02-25T15:12:55.614 DEBUG:teuthology.parallel:result is None
2026-02-25T15:12:55.628 INFO:teuthology.orchestra.run.trial145.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS"
2026-02-25T15:12:55.628 INFO:teuthology.orchestra.run.trial145.stdout:NAME="Ubuntu"
2026-02-25T15:12:55.628 INFO:teuthology.orchestra.run.trial145.stdout:VERSION_ID="22.04"
2026-02-25T15:12:55.628 INFO:teuthology.orchestra.run.trial145.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)"
2026-02-25T15:12:55.628 INFO:teuthology.orchestra.run.trial145.stdout:VERSION_CODENAME=jammy
2026-02-25T15:12:55.628 INFO:teuthology.orchestra.run.trial145.stdout:ID=ubuntu
2026-02-25T15:12:55.628 INFO:teuthology.orchestra.run.trial145.stdout:ID_LIKE=debian
2026-02-25T15:12:55.628 INFO:teuthology.orchestra.run.trial145.stdout:HOME_URL="https://www.ubuntu.com/"
2026-02-25T15:12:55.629 INFO:teuthology.orchestra.run.trial145.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2026-02-25T15:12:55.629 INFO:teuthology.orchestra.run.trial145.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2026-02-25T15:12:55.629 INFO:teuthology.orchestra.run.trial145.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2026-02-25T15:12:55.629 INFO:teuthology.orchestra.run.trial145.stdout:UBUNTU_CODENAME=jammy
2026-02-25T15:12:55.629 INFO:teuthology.provision.fog.trial145:Deploy complete!
2026-02-25T15:12:55.630 DEBUG:teuthology.parallel:result is None
2026-02-25T15:12:56.160 INFO:teuthology.lock.ops:Checking trial050.front.sepia.ceph.com
2026-02-25T15:12:56.160 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-25T15:12:56.259 INFO:teuthology.lock.ops:Checking trial145.front.sepia.ceph.com
2026-02-25T15:12:56.259 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-25T15:12:56.321 INFO:teuthology.lock.ops:Updating [trial050.front.sepia.ceph.com]: set os type and version on server
2026-02-25T15:12:56.321 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial050.front.sepia.ceph.com'
2026-02-25T15:12:56.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T15:12:56.383 DEBUG:teuthology.orchestra.run.trial050:> true
2026-02-25T15:12:56.460 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial050.front.sepia.ceph.com'
2026-02-25T15:12:56.460 DEBUG:teuthology.orchestra.run.trial050:> uname -m
2026-02-25T15:12:56.506 INFO:teuthology.orchestra.run.trial050.stdout:x86_64
2026-02-25T15:12:56.506 DEBUG:teuthology.orchestra.run.trial050:> cat /etc/os-release
2026-02-25T15:12:56.550 INFO:teuthology.orchestra.run.trial050.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS"
2026-02-25T15:12:56.551 INFO:teuthology.orchestra.run.trial050.stdout:NAME="Ubuntu"
2026-02-25T15:12:56.551 INFO:teuthology.orchestra.run.trial050.stdout:VERSION_ID="22.04"
2026-02-25T15:12:56.551 INFO:teuthology.orchestra.run.trial050.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)"
2026-02-25T15:12:56.551 INFO:teuthology.orchestra.run.trial050.stdout:VERSION_CODENAME=jammy
2026-02-25T15:12:56.551 INFO:teuthology.orchestra.run.trial050.stdout:ID=ubuntu
2026-02-25T15:12:56.551 INFO:teuthology.orchestra.run.trial050.stdout:ID_LIKE=debian
2026-02-25T15:12:56.551 INFO:teuthology.orchestra.run.trial050.stdout:HOME_URL="https://www.ubuntu.com/"
2026-02-25T15:12:56.551 INFO:teuthology.orchestra.run.trial050.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2026-02-25T15:12:56.551 INFO:teuthology.orchestra.run.trial050.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2026-02-25T15:12:56.551 INFO:teuthology.orchestra.run.trial050.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2026-02-25T15:12:56.552 INFO:teuthology.orchestra.run.trial050.stdout:UBUNTU_CODENAME=jammy
2026-02-25T15:12:56.552 INFO:teuthology.lock.ops:Updating trial050.front.sepia.ceph.com on lock server
2026-02-25T15:12:56.655 INFO:teuthology.lock.ops:Updating [trial145.front.sepia.ceph.com]: set os type and version on server
2026-02-25T15:12:56.655 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial145.front.sepia.ceph.com'
2026-02-25T15:12:56.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-25T15:12:56.725 DEBUG:teuthology.orchestra.run.trial145:> true
2026-02-25T15:12:56.802 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial145.front.sepia.ceph.com'
2026-02-25T15:12:56.802 DEBUG:teuthology.orchestra.run.trial145:> uname -m
2026-02-25T15:12:56.847 INFO:teuthology.orchestra.run.trial145.stdout:x86_64
2026-02-25T15:12:56.848 DEBUG:teuthology.orchestra.run.trial145:> cat /etc/os-release
2026-02-25T15:12:56.892 INFO:teuthology.orchestra.run.trial145.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS"
2026-02-25T15:12:56.892 INFO:teuthology.orchestra.run.trial145.stdout:NAME="Ubuntu"
2026-02-25T15:12:56.892 INFO:teuthology.orchestra.run.trial145.stdout:VERSION_ID="22.04"
2026-02-25T15:12:56.892 INFO:teuthology.orchestra.run.trial145.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)"
2026-02-25T15:12:56.892 INFO:teuthology.orchestra.run.trial145.stdout:VERSION_CODENAME=jammy
2026-02-25T15:12:56.892 INFO:teuthology.orchestra.run.trial145.stdout:ID=ubuntu
2026-02-25T15:12:56.892 INFO:teuthology.orchestra.run.trial145.stdout:ID_LIKE=debian
2026-02-25T15:12:56.892 INFO:teuthology.orchestra.run.trial145.stdout:HOME_URL="https://www.ubuntu.com/"
2026-02-25T15:12:56.892 INFO:teuthology.orchestra.run.trial145.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2026-02-25T15:12:56.893 INFO:teuthology.orchestra.run.trial145.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2026-02-25T15:12:56.893 INFO:teuthology.orchestra.run.trial145.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2026-02-25T15:12:56.893 INFO:teuthology.orchestra.run.trial145.stdout:UBUNTU_CODENAME=jammy
2026-02-25T15:12:56.893 INFO:teuthology.lock.ops:Updating trial145.front.sepia.ceph.com on lock server
2026-02-25T15:12:56.996 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:12:57.094 INFO:teuthology.dispatcher.supervisor:Running job 69601
2026-02-25T15:12:57.094 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/bin/teuthology -v --owner scheduled_vshankar@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69601 --name vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial --description fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/test_snap_schedule/{overrides/{v2} snap-schedule}} -- /home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69601/orig.config.yaml
2026-02-25T15:12:57.094 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69601
2026-02-25T15:12:57.094 INFO:teuthology.dispatcher.supervisor:Job PID: 2207989
2026-02-25T15:12:57.094 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-25T15:14:57.175 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:16:58.616 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:18:58.815 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:20:59.008 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:22:59.203 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:24:59.406 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:27:01.737 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:29:01.967 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:31:02.173 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:33:02.950 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:35:03.163 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:37:03.367 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:39:03.505 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:41:03.711 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:43:03.916 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:45:04.075 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:47:04.291 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:49:04.510 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:51:04.705 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:53:04.895 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:55:05.063 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:57:05.146 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T15:59:05.317 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:01:05.516 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:03:05.634 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:05:05.772 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:07:05.934 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:09:06.083 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:11:06.282 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:13:06.434 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:15:06.617 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:17:06.806 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:19:06.986 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:21:07.211 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:23:07.426 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:25:07.612 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:27:07.812 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:29:07.994 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:31:08.181 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:33:08.357 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:35:08.551 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:37:11.119 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:39:11.303 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:41:11.514 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:43:11.694 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:45:11.902 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:47:12.087 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:49:12.273 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:51:12.459 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:53:12.660 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:55:12.856 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:57:13.042 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T16:59:13.157 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T17:01:13.332 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T17:03:13.527 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T17:05:13.726 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T17:07:13.928 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T17:09:14.151 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T17:11:14.320 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T17:13:14.488 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T17:15:14.650 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T17:17:14.836 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T17:19:14.997 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T17:21:15.124 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T17:23:15.296 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T17:25:15.509 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T17:27:15.706 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T17:29:15.868 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-25T17:29:19.348 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-25T17:29:19.484 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-25T17:29:20.006 INFO:teuthology.orchestra.console.trial050:Power off
2026-02-25T17:29:20.007 DEBUG:teuthology.orchestra.console.trial050:pexpect command: ipmitool -H trial050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-25T17:29:20.028 INFO:teuthology.orchestra.console.trial145:Power off
2026-02-25T17:29:20.028 DEBUG:teuthology.orchestra.console.trial145:pexpect command: ipmitool -H trial145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-25T17:29:20.048 DEBUG:teuthology.orchestra.console.trial050:power off output: Chassis Power Control: Down/Off
2026-02-25T17:29:20.068 DEBUG:teuthology.orchestra.console.trial145:power off output: Chassis Power Control: Down/Off
2026-02-25T17:29:24.053 DEBUG:teuthology.orchestra.console.trial050:pexpect command: ipmitool -H trial050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T17:29:24.068 DEBUG:teuthology.orchestra.console.trial145:pexpect command: ipmitool -H trial145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T17:29:24.161 DEBUG:teuthology.orchestra.console.trial050:check power output: Chassis Power is on
2026-02-25T17:29:24.176 DEBUG:teuthology.orchestra.console.trial145:check power output: Chassis Power is on
2026-02-25T17:29:28.167 DEBUG:teuthology.orchestra.console.trial050:pexpect command: ipmitool -H trial050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T17:29:28.178 DEBUG:teuthology.orchestra.console.trial145:pexpect command: ipmitool -H trial145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-25T17:29:28.274 DEBUG:teuthology.orchestra.console.trial050:check power output: Chassis Power is off
2026-02-25T17:29:28.275 INFO:teuthology.orchestra.console.trial050:Power off completed
2026-02-25T17:29:28.286 DEBUG:teuthology.orchestra.console.trial145:check power output: Chassis Power is off
2026-02-25T17:29:28.286 INFO:teuthology.orchestra.console.trial145:Power off completed
2026-02-25T17:29:28.482 INFO:teuthology.lock.ops:unlocked: trial050.front.sepia.ceph.com
2026-02-25T17:29:28.495 INFO:teuthology.lock.ops:unlocked: trial145.front.sepia.ceph.com