2024-03-01T07:24:04.213 INFO:root:teuthology version: 0.0.1.dev249+gcd45576
2024-03-01T07:24:04.213 INFO:teuthology.results:subset: '419/1024'
2024-03-01T07:24:04.213 INFO:teuthology.results:seed: '5378'
2024-03-01T07:24:47.974 INFO:root:teuthology version: 0.0.1.dev249+gcd45576
2024-03-01T07:24:47.978 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2024-03-01T08:27:52.523 INFO:teuthology.results:Tests finished! gathering results...
2024-03-01T08:27:52.647 INFO:scrape:Found 1 jobs
2024-03-01T08:27:52.879 WARNING:scrape:Saw end of BT but not start
2024-03-01T08:27:52.879 WARNING:scrape:Saw end of BT but not start
2024-03-01T08:27:52.879 WARNING:scrape:Saw end of BT but not start
2024-03-01T08:27:52.879 WARNING:scrape:Saw end of BT but not start
2024-03-01T08:27:53.006 WARNING:scrape:Saw end of BT but not start
2024-03-01T08:27:53.007 WARNING:scrape:Saw end of BT but not start
2024-03-01T08:27:53.007 WARNING:scrape:Saw end of BT but not start
2024-03-01T08:27:53.007 WARNING:scrape:Saw end of BT but not start
2024-03-01T08:27:53.132 WARNING:scrape:Saw end of BT but not start
2024-03-01T08:27:53.132 WARNING:scrape:Saw end of BT but not start
2024-03-01T08:27:53.132 WARNING:scrape:Saw end of BT but not start
2024-03-01T08:27:53.132 WARNING:scrape:Saw end of BT but not start
2024-03-01T08:27:53.180 INFO:scrape:Found 1 distinct failure reasons
2024-03-01T08:27:53.180 INFO:scrape:Failure: error during scrub thrashing: rank damage found: {'backtrace'}
2024-03-01T08:27:53.180 INFO:scrape:1 jobs: ['7577479']
2024-03-01T08:27:53.180 INFO:scrape:suites: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/yes', '5-workunit/suites/iogen}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs:workload/{0-centos_9.stream', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'n/3', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd', 'osd-asserts', 'overrides/{cephsqlite-timeout', 'ranks/multi/{balancer/automatic', 'replication/always}', 'selinux}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-namespace-isolated-and-quota}']
2024-03-01T08:27:53.180 INFO:scrape:
2024-03-01T08:27:53.181 INFO:teuthology.results:Sending results to vshankar@redhat.com: Test Run: vshankar-2024-03-01_07:23:58-fs:workload-wip-vshankar-testing-20240226.064629-testing-default-smithi
=================================================================
info:    https://pulpito.ceph.com/vshankar-2024-03-01_07:23:58-fs:workload-wip-vshankar-testing-20240226.064629-testing-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/vshankar-2024-03-01_07:23:58-fs:workload-wip-vshankar-testing-20240226.064629-testing-default-smithi/
failed:  1
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[7577479]  fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd selinux} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/iogen}}
-----------------------------------------------------------------
time:   00:52:55
info:   https://pulpito.ceph.com/vshankar-2024-03-01_07:23:58-fs:workload-wip-vshankar-testing-20240226.064629-testing-default-smithi/7577479/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2024-03-01_07:23:58-fs:workload-wip-vshankar-testing-20240226.064629-testing-default-smithi/7577479/

    error during scrub thrashing: rank damage found: {'backtrace'}