2026-02-24T06:55:43.312 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-24T06:55:43.313 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T06:55:43.388 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T06:55:43.583 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7a5f6df094c0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7a5f6de9cb80>, signals=[15])
2026-02-24T06:55:43.583 INFO:teuthology.lock.ops:Start node 'trial102.front.sepia.ceph.com' reimaging
2026-02-24T06:55:43.583 INFO:teuthology.lock.ops:Updating [trial102.front.sepia.ceph.com]: reset os type and version on server
2026-02-24T06:55:43.583 INFO:teuthology.lock.ops:Updating trial102.front.sepia.ceph.com on lock server
2026-02-24T06:55:43.641 INFO:teuthology.lock.ops:Start node 'trial122.front.sepia.ceph.com' reimaging
2026-02-24T06:55:43.641 INFO:teuthology.lock.ops:Updating [trial122.front.sepia.ceph.com]: reset os type and version on server
2026-02-24T06:55:43.641 INFO:teuthology.lock.ops:Updating trial122.front.sepia.ceph.com on lock server
2026-02-24T06:55:43.858 DEBUG:teuthology.provision.fog.trial102:Requesting image trial_centos_9.stream (ID 3)
2026-02-24T06:55:43.882 DEBUG:teuthology.provision.fog.trial122:Requesting image trial_centos_9.stream (ID 3)
2026-02-24T06:55:43.910 INFO:teuthology.provision.fog.trial102:Scheduling deploy of centos 9.stream
2026-02-24T06:55:43.935 INFO:teuthology.provision.fog.trial122:Scheduling deploy of centos 9.stream
2026-02-24T06:55:45.238 INFO:teuthology.orchestra.console.trial102:Power off
2026-02-24T06:55:45.238 DEBUG:teuthology.orchestra.console.trial102:pexpect command: ipmitool -H trial102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T06:55:45.259 INFO:teuthology.orchestra.console.trial122:Power off
2026-02-24T06:55:45.259 DEBUG:teuthology.orchestra.console.trial122:pexpect command: ipmitool -H trial122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T06:55:45.276 DEBUG:teuthology.orchestra.console.trial102:power off output: Chassis Power Control: Down/Off
2026-02-24T06:55:45.297 DEBUG:teuthology.orchestra.console.trial122:power off output: Chassis Power Control: Down/Off
2026-02-24T06:55:49.282 DEBUG:teuthology.orchestra.console.trial102:pexpect command: ipmitool -H trial102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T06:55:49.298 DEBUG:teuthology.orchestra.console.trial122:pexpect command: ipmitool -H trial122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T06:55:49.392 DEBUG:teuthology.orchestra.console.trial102:check power output: Chassis Power is off
2026-02-24T06:55:49.392 INFO:teuthology.orchestra.console.trial102:Power off completed
2026-02-24T06:55:49.405 DEBUG:teuthology.orchestra.console.trial122:check power output: Chassis Power is off
2026-02-24T06:55:49.405 INFO:teuthology.orchestra.console.trial122:Power off completed
2026-02-24T06:55:49.494 INFO:teuthology.orchestra.console.trial102:Power on
2026-02-24T06:55:49.494 DEBUG:teuthology.orchestra.console.trial102:pexpect command: ipmitool -H trial102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-24T06:55:49.506 INFO:teuthology.orchestra.console.trial122:Power on
2026-02-24T06:55:49.507 DEBUG:teuthology.orchestra.console.trial122:pexpect command: ipmitool -H trial122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-24T06:55:49.534 DEBUG:teuthology.orchestra.console.trial102:power on output: Chassis Power Control: Up/On
2026-02-24T06:55:49.548 DEBUG:teuthology.orchestra.console.trial122:power on output: Chassis Power Control: Up/On
2026-02-24T06:55:53.539 DEBUG:teuthology.orchestra.console.trial102:pexpect command: ipmitool -H trial102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T06:55:53.548 DEBUG:teuthology.orchestra.console.trial122:pexpect command: ipmitool -H trial122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T06:55:53.651 DEBUG:teuthology.orchestra.console.trial102:check power output: Chassis Power is on
2026-02-24T06:55:53.651 INFO:teuthology.orchestra.console.trial102:Power on completed
2026-02-24T06:55:53.658 DEBUG:teuthology.orchestra.console.trial122:check power output: Chassis Power is on
2026-02-24T06:55:53.658 INFO:teuthology.orchestra.console.trial122:Power on completed
2026-02-24T06:55:53.752 INFO:teuthology.provision.fog.trial102:Waiting for deploy to finish
2026-02-24T06:55:53.760 INFO:teuthology.provision.fog.trial122:Waiting for deploy to finish
2026-02-24T06:57:56.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T06:57:56.940 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T06:58:11.200 WARNING:teuthology.provision.fog.trial102:[Errno None] Unable to connect to port 22 on 10.20.193.102
2026-02-24T06:58:11.200 WARNING:teuthology.provision.fog.trial122:[Errno None] Unable to connect to port 22 on 10.20.193.122
2026-02-24T06:58:17.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T06:58:17.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T06:58:20.288 WARNING:teuthology.provision.fog.trial102:[Errno None] Unable to connect to port 22 on 10.20.193.102
2026-02-24T06:58:20.288 WARNING:teuthology.provision.fog.trial122:[Errno None] Unable to connect to port 22 on 10.20.193.122
2026-02-24T06:58:26.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T06:58:26.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T06:58:29.376 WARNING:teuthology.provision.fog.trial102:[Errno None] Unable to connect to port 22 on 10.20.193.102
2026-02-24T06:58:29.376 WARNING:teuthology.provision.fog.trial122:[Errno None] Unable to connect to port 22 on 10.20.193.122
2026-02-24T06:58:35.383 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T06:58:35.385 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T06:59:11.524 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-24T06:59:11.524 DEBUG:teuthology.orchestra.run.trial122:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-24T06:59:11.525 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-24T06:59:11.525 DEBUG:teuthology.orchestra.run.trial102:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-24T06:59:11.675 INFO:teuthology.provision.fog.trial102:Node is ready
2026-02-24T06:59:11.676 DEBUG:teuthology.orchestra.run.trial102:> hostname
2026-02-24T06:59:11.680 INFO:teuthology.provision.fog.trial122:Node is ready
2026-02-24T06:59:11.680 DEBUG:teuthology.orchestra.run.trial122:> hostname
2026-02-24T06:59:11.728 INFO:teuthology.orchestra.run.trial102.stdout:trial102.front.sepia.ceph.com
2026-02-24T06:59:11.729 DEBUG:teuthology.orchestra.run.trial102:> grep trial102.front.sepia.ceph.com /etc/hosts
2026-02-24T06:59:11.736 INFO:teuthology.orchestra.run.trial122.stdout:trial122.front.sepia.ceph.com
2026-02-24T06:59:11.736 DEBUG:teuthology.orchestra.run.trial122:> grep trial122.front.sepia.ceph.com /etc/hosts
2026-02-24T06:59:11.782 INFO:teuthology.orchestra.run.trial102.stdout:10.20.193.102 trial102.front.sepia.ceph.com trial102
2026-02-24T06:59:11.782 DEBUG:teuthology.orchestra.run.trial102:> sudo hostname trial102
2026-02-24T06:59:11.792 INFO:teuthology.orchestra.run.trial122.stdout:10.20.193.122 trial122.front.sepia.ceph.com trial122
2026-02-24T06:59:11.793 DEBUG:teuthology.orchestra.run.trial122:> sudo hostname trial122
2026-02-24T06:59:11.849 DEBUG:teuthology.orchestra.run.trial102:> sudo sed -i -e 's/trial102.front.sepia.ceph.com/trial102/g' /etc/hosts
2026-02-24T06:59:11.862 DEBUG:teuthology.orchestra.run.trial122:> sudo sed -i -e 's/trial122.front.sepia.ceph.com/trial122/g' /etc/hosts
2026-02-24T06:59:11.913 DEBUG:teuthology.orchestra.run.trial102:> sudo sed -i -e 's/10.20.193.102/10.20.193.102/g' /etc/hosts
2026-02-24T06:59:11.927 DEBUG:teuthology.orchestra.run.trial122:> sudo sed -i -e 's/10.20.193.122/10.20.193.122/g' /etc/hosts
2026-02-24T06:59:11.975 DEBUG:teuthology.orchestra.run.trial102:> sudo sed -i -e 's/trial102.front.sepia.ceph.com/trial102/g' /etc/hostname
2026-02-24T06:59:11.991 DEBUG:teuthology.orchestra.run.trial122:> sudo sed -i -e 's/trial122.front.sepia.ceph.com/trial122/g' /etc/hostname
2026-02-24T06:59:12.036 DEBUG:teuthology.orchestra.run.trial102:> sudo hostname trial102
2026-02-24T06:59:12.056 DEBUG:teuthology.orchestra.run.trial122:> sudo hostname trial122
2026-02-24T06:59:12.097 DEBUG:teuthology.orchestra.run.trial102:> cat /etc/os-release
2026-02-24T06:59:12.120 DEBUG:teuthology.orchestra.run.trial122:> cat /etc/os-release
2026-02-24T06:59:12.151 INFO:teuthology.orchestra.run.trial102.stdout:NAME="CentOS Stream"
2026-02-24T06:59:12.151 INFO:teuthology.orchestra.run.trial102.stdout:VERSION="9"
2026-02-24T06:59:12.152 INFO:teuthology.orchestra.run.trial102.stdout:ID="centos"
2026-02-24T06:59:12.152 INFO:teuthology.orchestra.run.trial102.stdout:ID_LIKE="rhel fedora"
2026-02-24T06:59:12.152 INFO:teuthology.orchestra.run.trial102.stdout:VERSION_ID="9"
2026-02-24T06:59:12.152 INFO:teuthology.orchestra.run.trial102.stdout:PLATFORM_ID="platform:el9"
2026-02-24T06:59:12.152 INFO:teuthology.orchestra.run.trial102.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T06:59:12.152 INFO:teuthology.orchestra.run.trial102.stdout:ANSI_COLOR="0;31"
2026-02-24T06:59:12.152 INFO:teuthology.orchestra.run.trial102.stdout:LOGO="fedora-logo-icon"
2026-02-24T06:59:12.152 INFO:teuthology.orchestra.run.trial102.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T06:59:12.152 INFO:teuthology.orchestra.run.trial102.stdout:HOME_URL="https://centos.org/"
2026-02-24T06:59:12.152 INFO:teuthology.orchestra.run.trial102.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T06:59:12.152 INFO:teuthology.orchestra.run.trial102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T06:59:12.152 INFO:teuthology.orchestra.run.trial102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T06:59:12.153 INFO:teuthology.provision.fog.trial102:Deploy complete!
2026-02-24T06:59:12.155 DEBUG:teuthology.parallel:result is None
2026-02-24T06:59:12.177 INFO:teuthology.orchestra.run.trial122.stdout:NAME="CentOS Stream"
2026-02-24T06:59:12.177 INFO:teuthology.orchestra.run.trial122.stdout:VERSION="9"
2026-02-24T06:59:12.177 INFO:teuthology.orchestra.run.trial122.stdout:ID="centos"
2026-02-24T06:59:12.177 INFO:teuthology.orchestra.run.trial122.stdout:ID_LIKE="rhel fedora"
2026-02-24T06:59:12.177 INFO:teuthology.orchestra.run.trial122.stdout:VERSION_ID="9"
2026-02-24T06:59:12.177 INFO:teuthology.orchestra.run.trial122.stdout:PLATFORM_ID="platform:el9"
2026-02-24T06:59:12.177 INFO:teuthology.orchestra.run.trial122.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T06:59:12.177 INFO:teuthology.orchestra.run.trial122.stdout:ANSI_COLOR="0;31"
2026-02-24T06:59:12.178 INFO:teuthology.orchestra.run.trial122.stdout:LOGO="fedora-logo-icon"
2026-02-24T06:59:12.178 INFO:teuthology.orchestra.run.trial122.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T06:59:12.178 INFO:teuthology.orchestra.run.trial122.stdout:HOME_URL="https://centos.org/"
2026-02-24T06:59:12.178 INFO:teuthology.orchestra.run.trial122.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T06:59:12.178 INFO:teuthology.orchestra.run.trial122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T06:59:12.178 INFO:teuthology.orchestra.run.trial122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T06:59:12.178 INFO:teuthology.provision.fog.trial122:Deploy complete!
2026-02-24T06:59:12.179 DEBUG:teuthology.parallel:result is None
2026-02-24T06:59:12.576 INFO:teuthology.lock.ops:Checking trial102.front.sepia.ceph.com
2026-02-24T06:59:12.576 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-24T06:59:12.680 INFO:teuthology.lock.ops:Checking trial122.front.sepia.ceph.com
2026-02-24T06:59:12.680 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-24T06:59:12.743 INFO:teuthology.lock.ops:Updating [trial102.front.sepia.ceph.com]: set os type and version on server
2026-02-24T06:59:12.743 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial102.front.sepia.ceph.com'
2026-02-24T06:59:12.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T06:59:12.804 DEBUG:teuthology.orchestra.run.trial102:> true
2026-02-24T06:59:12.881 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial102.front.sepia.ceph.com'
2026-02-24T06:59:12.881 DEBUG:teuthology.orchestra.run.trial102:> uname -m
2026-02-24T06:59:12.937 INFO:teuthology.orchestra.run.trial102.stdout:x86_64
2026-02-24T06:59:12.937 DEBUG:teuthology.orchestra.run.trial102:> cat /etc/os-release
2026-02-24T06:59:12.993 INFO:teuthology.orchestra.run.trial102.stdout:NAME="CentOS Stream"
2026-02-24T06:59:12.993 INFO:teuthology.orchestra.run.trial102.stdout:VERSION="9"
2026-02-24T06:59:12.993 INFO:teuthology.orchestra.run.trial102.stdout:ID="centos"
2026-02-24T06:59:12.993 INFO:teuthology.orchestra.run.trial102.stdout:ID_LIKE="rhel fedora"
2026-02-24T06:59:12.993 INFO:teuthology.orchestra.run.trial102.stdout:VERSION_ID="9"
2026-02-24T06:59:12.993 INFO:teuthology.orchestra.run.trial102.stdout:PLATFORM_ID="platform:el9"
2026-02-24T06:59:12.993 INFO:teuthology.orchestra.run.trial102.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T06:59:12.993 INFO:teuthology.orchestra.run.trial102.stdout:ANSI_COLOR="0;31"
2026-02-24T06:59:12.993 INFO:teuthology.orchestra.run.trial102.stdout:LOGO="fedora-logo-icon"
2026-02-24T06:59:12.994 INFO:teuthology.orchestra.run.trial102.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T06:59:12.994 INFO:teuthology.orchestra.run.trial102.stdout:HOME_URL="https://centos.org/"
2026-02-24T06:59:12.994 INFO:teuthology.orchestra.run.trial102.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T06:59:12.994 INFO:teuthology.orchestra.run.trial102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T06:59:12.994 INFO:teuthology.orchestra.run.trial102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T06:59:12.995 INFO:teuthology.lock.ops:Updating trial102.front.sepia.ceph.com on lock server
2026-02-24T06:59:13.097 INFO:teuthology.lock.ops:Updating [trial122.front.sepia.ceph.com]: set os type and version on server
2026-02-24T06:59:13.097 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial122.front.sepia.ceph.com'
2026-02-24T06:59:13.097 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T06:59:13.166 DEBUG:teuthology.orchestra.run.trial122:> true
2026-02-24T06:59:13.245 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial122.front.sepia.ceph.com'
2026-02-24T06:59:13.245 DEBUG:teuthology.orchestra.run.trial122:> uname -m
2026-02-24T06:59:13.301 INFO:teuthology.orchestra.run.trial122.stdout:x86_64
2026-02-24T06:59:13.301 DEBUG:teuthology.orchestra.run.trial122:> cat /etc/os-release
2026-02-24T06:59:13.357 INFO:teuthology.orchestra.run.trial122.stdout:NAME="CentOS Stream"
2026-02-24T06:59:13.357 INFO:teuthology.orchestra.run.trial122.stdout:VERSION="9"
2026-02-24T06:59:13.357 INFO:teuthology.orchestra.run.trial122.stdout:ID="centos"
2026-02-24T06:59:13.357 INFO:teuthology.orchestra.run.trial122.stdout:ID_LIKE="rhel fedora"
2026-02-24T06:59:13.357 INFO:teuthology.orchestra.run.trial122.stdout:VERSION_ID="9"
2026-02-24T06:59:13.357 INFO:teuthology.orchestra.run.trial122.stdout:PLATFORM_ID="platform:el9"
2026-02-24T06:59:13.358 INFO:teuthology.orchestra.run.trial122.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-24T06:59:13.358 INFO:teuthology.orchestra.run.trial122.stdout:ANSI_COLOR="0;31"
2026-02-24T06:59:13.358 INFO:teuthology.orchestra.run.trial122.stdout:LOGO="fedora-logo-icon"
2026-02-24T06:59:13.358 INFO:teuthology.orchestra.run.trial122.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-24T06:59:13.358 INFO:teuthology.orchestra.run.trial122.stdout:HOME_URL="https://centos.org/"
2026-02-24T06:59:13.358 INFO:teuthology.orchestra.run.trial122.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-24T06:59:13.358 INFO:teuthology.orchestra.run.trial122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-24T06:59:13.358 INFO:teuthology.orchestra.run.trial122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-24T06:59:13.359 INFO:teuthology.lock.ops:Updating trial122.front.sepia.ceph.com on lock server
2026-02-24T06:59:13.461 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T06:59:13.560 INFO:teuthology.dispatcher.supervisor:Running job 67549
2026-02-24T06:59:13.560 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/bin/teuthology -v --owner scheduled_shraddhaag@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67549 --name shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial --description crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/cephadm objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/{default} thrashosds-health workloads/snaps-few-objects-balanced} -- /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67549/orig.config.yaml
2026-02-24T06:59:13.560 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67549
2026-02-24T06:59:13.560 INFO:teuthology.dispatcher.supervisor:Job PID: 2678050
2026-02-24T06:59:13.560 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-24T07:01:13.640 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:03:13.831 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:05:14.027 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:07:14.211 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:09:14.398 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:11:14.589 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:13:14.785 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:15:14.998 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:17:15.124 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:19:15.342 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:21:15.531 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:23:16.657 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:25:16.867 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:27:17.017 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:29:17.231 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:31:23.795 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:33:23.979 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:35:24.186 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:37:24.361 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:39:24.563 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:41:24.750 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:43:24.934 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:45:25.129 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:47:25.311 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:49:25.506 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:51:25.693 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:53:25.881 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:55:26.066 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:57:26.232 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T07:59:26.438 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:01:26.552 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:03:26.743 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:05:26.912 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:07:27.120 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:09:27.451 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:11:27.621 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:13:27.805 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:15:28.152 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:17:28.368 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:19:28.579 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:21:28.786 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:23:28.988 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:25:29.124 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:27:29.316 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:29:29.517 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:31:29.715 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:33:29.926 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:35:30.135 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:37:30.307 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:39:30.494 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:41:30.678 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:43:30.884 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:45:31.090 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:47:31.282 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:49:31.461 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:51:31.644 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:53:31.848 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:55:32.019 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:57:32.219 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T08:59:32.399 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:01:32.599 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:03:32.789 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:05:33.072 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:07:33.279 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:09:33.486 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:11:33.700 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:13:33.986 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:15:34.161 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:17:34.380 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:19:34.535 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:21:34.728 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:23:34.911 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:25:35.120 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:27:35.333 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:29:35.500 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:31:35.707 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:33:35.826 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:35:36.021 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:37:36.226 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:39:36.399 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:41:36.588 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:43:36.765 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:45:36.967 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:47:37.171 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:49:37.361 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:51:37.553 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:53:37.756 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:55:37.890 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:57:38.084 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T09:59:38.258 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:01:38.402 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:03:38.600 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:05:38.818 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:07:39.010 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:09:39.181 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:11:39.384 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:13:39.571 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:15:39.678 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:17:39.858 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:19:40.068 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:21:40.267 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:23:40.478 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:25:40.578 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:27:40.802 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:29:40.996 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:31:41.189 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:33:41.372 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:35:41.556 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:37:41.779 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:39:41.972 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:41:42.177 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:43:42.366 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:45:42.563 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:47:42.785 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:49:42.966 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:51:43.172 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:53:43.362 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:55:43.545 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:57:43.714 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T10:59:43.855 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:01:44.056 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:03:44.231 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:05:44.422 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:07:44.599 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:09:44.809 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:11:45.019 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:13:45.208 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:15:45.423 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:17:45.569 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:19:45.771 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:21:45.927 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:23:46.255 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:25:46.461 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:27:46.655 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:29:46.775 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:31:48.754 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:33:48.944 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:35:49.282 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:37:49.421 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:39:51.787 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:41:52.003 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:43:52.200 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:45:52.364 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:47:52.527 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:49:52.742 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:51:52.939 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:53:53.083 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:55:53.294 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:57:53.596 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T11:59:53.811 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:01:54.054 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:03:54.267 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:05:54.475 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:07:54.661 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:09:54.749 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:11:54.847 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:13:55.036 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:15:55.238 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:17:55.439 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:19:55.617 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:21:55.740 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:23:55.916 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:25:56.084 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:27:56.278 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:29:56.460 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:31:56.643 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:33:56.862 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:35:57.020 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:37:57.202 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:39:57.423 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:41:57.620 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:43:57.851 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:45:58.062 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:47:58.250 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:49:58.365 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:51:58.506 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:53:58.726 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:55:58.940 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:57:59.120 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:59:59.304 WARNING:teuthology.dispatcher.supervisor:Job shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67549 ran longer than 21600s. Killing...
2026-02-24T12:59:59.393 INFO:teuthology.kill:Killing Pids: {2678050}
2026-02-24T12:59:59.394 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T12:59:59.499 INFO:teuthology.task.internal:roles: ubuntu@trial102.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a']
2026-02-24T12:59:59.499 INFO:teuthology.task.internal:roles: ubuntu@trial122.front.sepia.ceph.com - ['osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']
2026-02-24T12:59:59.499 INFO:teuthology.misc:Compressing logs...
2026-02-24T12:59:59.499 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial102.front.sepia.ceph.com'
2026-02-24T12:59:59.500 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T12:59:59.563 DEBUG:teuthology.orchestra.run.trial102:> true
2026-02-24T12:59:59.635 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial102.front.sepia.ceph.com'
2026-02-24T12:59:59.635 DEBUG:teuthology.orchestra.run.trial102:> sudo find /var/lib/ceph/crash -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --
2026-02-24T12:59:59.677 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial122.front.sepia.ceph.com'
2026-02-24T12:59:59.677 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-24T12:59:59.759 DEBUG:teuthology.orchestra.run.trial122:> true
2026-02-24T12:59:59.840 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial122.front.sepia.ceph.com'
2026-02-24T12:59:59.840 DEBUG:teuthology.orchestra.run.trial122:> sudo find /var/lib/ceph/crash -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --
2026-02-24T12:59:59.913 DEBUG:teuthology.misc:Transferring archived files from trial102:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67549/remote/trial102/crash
2026-02-24T12:59:59.914 DEBUG:teuthology.orchestra.run.trial102:> sudo tar c -f - -C /var/lib/ceph/crash -- .
2026-02-24T12:59:59.941 DEBUG:teuthology.misc:Transferring archived files from trial122:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67549/remote/trial122/crash
2026-02-24T12:59:59.943 DEBUG:teuthology.orchestra.run.trial122:> sudo tar c -f - -C /var/lib/ceph/crash -- .
2026-02-24T12:59:59.973 INFO:teuthology.misc:Compressing logs...
2026-02-24T12:59:59.973 DEBUG:teuthology.orchestra.run.trial102:> sudo find /home/ubuntu/cephtest/archive -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --
2026-02-24T12:59:59.986 DEBUG:teuthology.orchestra.run.trial122:> sudo find /home/ubuntu/cephtest/archive -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --
2026-02-24T13:00:00.019 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
2026-02-24T13:00:00.019 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
2026-02-24T13:00:00.020 INFO:teuthology.orchestra.run.trial102.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log:	  0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2026-02-24T13:00:00.020 INFO:teuthology.orchestra.run.trial102.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log:	  0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2026-02-24T13:00:00.059 INFO:teuthology.orchestra.run.trial122.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
2026-02-24T13:00:00.059 INFO:teuthology.orchestra.run.trial122.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
2026-02-24T13:00:00.060 INFO:teuthology.orchestra.run.trial122.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log:	  0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2026-02-24T13:00:00.060 INFO:teuthology.orchestra.run.trial122.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log:	  0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2026-02-24T13:00:00.062 DEBUG:teuthology.misc:Transferring archived files from trial102:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67549/remote/trial102/
2026-02-24T13:00:00.062 DEBUG:teuthology.orchestra.run.trial102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- .
2026-02-24T13:00:00.090 DEBUG:teuthology.misc:Transferring archived files from trial122:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67549/remote/trial122/
2026-02-24T13:00:00.090 DEBUG:teuthology.orchestra.run.trial122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- .
2026-02-24T13:00:00.133 INFO:teuthology.misc:Compressing logs...
2026-02-24T13:00:00.133 DEBUG:teuthology.orchestra.run.trial102:> sudo find /var/log/ceph -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --
2026-02-24T13:00:00.135 DEBUG:teuthology.orchestra.run.trial122:> sudo find /var/log/ceph -name *.log -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --
2026-02-24T13:00:00.156 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log
2026-02-24T13:00:00.157 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-mon.a.log
2026-02-24T13:00:00.159 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/cephadm.log:	gzip -5 --verbose -- /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-osd.0.log
2026-02-24T13:00:00.159 INFO:teuthology.orchestra.run.trial102.stderr:gzip -5 --verbose -- /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph.cephadm.log
2026-02-24T13:00:00.159 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-mon.a.log:	/var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-osd.0.log:	gzip -5 --verbose -- /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph.audit.log
2026-02-24T13:00:00.160 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph.cephadm.log:	gzip -5 --verbose -- /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-volume.log
2026-02-24T13:00:00.160 INFO:teuthology.orchestra.run.trial102.stderr: 82.1% -- replaced with /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph.cephadm.log.gz
2026-02-24T13:00:00.160 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph.audit.log:	gzip -5 --verbose -- /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-osd.1.log
2026-02-24T13:00:00.160 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-volume.log:	gzip -5 --verbose -- /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-mgr.x.log
2026-02-24T13:00:00.160 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-osd.1.log:	gzip -5 --verbose -- /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph.log
2026-02-24T13:00:00.162 INFO:teuthology.orchestra.run.trial102.stderr:/var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-mgr.x.log:	/var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph.log:	 95.0% -- replaced with /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-volume.log.gz
2026-02-24T13:00:00.167 INFO:teuthology.orchestra.run.trial102.stderr: 90.5% -- replaced with /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph.audit.log.gz
2026-02-24T13:00:00.173 INFO:teuthology.orchestra.run.trial102.stderr: 91.8% -- replaced with /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph.log.gz
2026-02-24T13:00:00.196 INFO:teuthology.orchestra.run.trial102.stderr: 97.0% -- replaced with /var/log/ceph/cephadm.log.gz
2026-02-24T13:00:00.197 INFO:teuthology.orchestra.run.trial122.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log
2026-02-24T13:00:00.199 INFO:teuthology.orchestra.run.trial122.stderr:gzip -5 --verbose -- /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-osd.2.log
2026-02-24T13:00:00.199 INFO:teuthology.orchestra.run.trial122.stderr:/var/log/ceph/cephadm.log:	gzip -5 --verbose -- /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-osd.3.log
2026-02-24T13:00:00.199 INFO:teuthology.orchestra.run.trial122.stderr:gzip -5 --verbose -- /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-volume.log
2026-02-24T13:00:00.200 INFO:teuthology.orchestra.run.trial122.stderr:/var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-osd.2.log:	/var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-osd.3.log:	/var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-volume.log:	 86.9% -- replaced with /var/log/ceph/cephadm.log.gz
2026-02-24T13:00:00.203 INFO:teuthology.orchestra.run.trial122.stderr: 95.1% -- replaced with /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-volume.log.gz
2026-02-24T13:00:00.794 INFO:teuthology.orchestra.run.trial102.stderr:gzip: /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-mgr.x.log: file size changed while zipping
2026-02-24T13:00:00.794 INFO:teuthology.orchestra.run.trial102.stderr: 92.5% -- replaced with /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-mgr.x.log.gz
2026-02-24T13:00:01.289 INFO:teuthology.orchestra.run.trial122.stderr: 94.6% -- replaced with /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-osd.3.log.gz
2026-02-24T13:00:04.181 INFO:teuthology.orchestra.run.trial102.stderr:gzip: /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-mon.a.log: file size changed while zipping
2026-02-24T13:00:04.181 INFO:teuthology.orchestra.run.trial102.stderr: 88.8% -- replaced with /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-mon.a.log.gz
2026-02-24T13:00:05.437 INFO:teuthology.orchestra.run.trial102.stderr:gzip: /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-osd.0.log: file size changed while zipping
2026-02-24T13:00:05.437 INFO:teuthology.orchestra.run.trial102.stderr: 94.5% -- replaced with /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-osd.0.log.gz
2026-02-24T13:00:06.139 INFO:teuthology.orchestra.run.trial122.stderr:gzip: /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-osd.2.log: file size changed while zipping
2026-02-24T13:00:06.139 INFO:teuthology.orchestra.run.trial122.stderr: 94.5% -- replaced with /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-osd.2.log.gz
2026-02-24T13:00:07.649 INFO:teuthology.orchestra.run.trial102.stderr:gzip: /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-osd.1.log: file size changed while zipping
2026-02-24T13:00:07.649 INFO:teuthology.orchestra.run.trial102.stderr: 94.6% -- replaced with /var/log/ceph/cfd7bd45-1152-11f1-8640-d404e6e7d460/ceph-osd.1.log.gz
2026-02-24T13:00:07.651 DEBUG:teuthology.misc:Transferring archived files from trial102:/var/log/ceph to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67549/remote/trial102/log
2026-02-24T13:00:07.653 DEBUG:teuthology.orchestra.run.trial102:> sudo tar c -f - -C /var/log/ceph -- .
2026-02-24T13:00:09.077 DEBUG:teuthology.misc:Transferring archived files from trial122:/var/log/ceph to /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-24_06:51:01-crimson-rados-main-distro-debug-trial/67549/remote/trial122/log
2026-02-24T13:00:09.079 DEBUG:teuthology.orchestra.run.trial122:> sudo tar c -f - -C /var/log/ceph -- .
2026-02-24T13:00:09.671 INFO:teuthology.kill:No teuthology processes running
2026-02-24T13:00:09.671 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T13:00:10.324 INFO:teuthology.orchestra.console.trial102:Power off
2026-02-24T13:00:10.324 DEBUG:teuthology.orchestra.console.trial102:pexpect command: ipmitool -H trial102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T13:00:10.342 INFO:teuthology.orchestra.console.trial122:Power off
2026-02-24T13:00:10.343 DEBUG:teuthology.orchestra.console.trial122:pexpect command: ipmitool -H trial122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-24T13:00:10.363 DEBUG:teuthology.orchestra.console.trial102:power off output: Chassis Power Control: Down/Off
2026-02-24T13:00:10.381 DEBUG:teuthology.orchestra.console.trial122:power off output: Chassis Power Control: Down/Off
2026-02-24T13:00:14.367 DEBUG:teuthology.orchestra.console.trial102:pexpect command: ipmitool -H trial102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T13:00:14.382 DEBUG:teuthology.orchestra.console.trial122:pexpect command: ipmitool -H trial122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T13:00:14.477 DEBUG:teuthology.orchestra.console.trial102:check power output: Chassis Power is on
2026-02-24T13:00:14.493 DEBUG:teuthology.orchestra.console.trial122:check power output: Chassis Power is on
2026-02-24T13:00:18.481 DEBUG:teuthology.orchestra.console.trial102:pexpect command: ipmitool -H trial102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T13:00:18.494 DEBUG:teuthology.orchestra.console.trial122:pexpect command: ipmitool -H trial122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-24T13:00:18.591 DEBUG:teuthology.orchestra.console.trial102:check power output: Chassis Power is off
2026-02-24T13:00:18.591 INFO:teuthology.orchestra.console.trial102:Power off completed
2026-02-24T13:00:18.603 DEBUG:teuthology.orchestra.console.trial122:check power output: Chassis Power is off
2026-02-24T13:00:18.603 INFO:teuthology.orchestra.console.trial122:Power off completed
2026-02-24T13:00:18.826 INFO:teuthology.lock.ops:unlocked: trial102.front.sepia.ceph.com
2026-02-24T13:00:18.839 INFO:teuthology.lock.ops:unlocked: trial122.front.sepia.ceph.com
2026-02-24T13:00:18.841 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T13:02:19.010 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-24T13:02:19.082 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2026-02-24T13:02:19.198 WARNING:teuthology.dispatcher.supervisor:Was going to unlock trial102 but it was locked by another job: /home/teuthworker/mnt/teuthology/nithyab-2026-02-24_12:43:35-rgw-wip-nbalacha-lua-74219-distro-default-trial/67993
2026-02-24T13:02:19.198 WARNING:teuthology.dispatcher.supervisor:Was going to unlock trial122 but it was locked by another job: /home/teuthworker/mnt/teuthology/nithyab-2026-02-24_12:43:35-rgw-wip-nbalacha-lua-74219-distro-default-trial/67993