Found 33 jobs
Missing teuthology log /home/teuthworker/archive/pdonnell-2021-07-02_10:08:50-fs-wip-pdonnell-testing-20210701.192056-distro-basic-smithi/6249130/teuthology.log
Found 5 distinct failure reasons
Timeout 3h running clone.client.0/qa/workunits/fs/misc/multiple_rsync.sh
1 jobs: ['6249140']
suites: ['clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/workload/{begin', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-bitmap', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{frag_enable', 'ranks/5', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/misc}', 'wsync/{no}}']

Failure: Test failure: test_barrier (tasks.cephfs.test_full.TestClusterFull)
2 jobs: ['6249132', '6249157']
suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'osd}', 'overrides/{frag_enable', 'tasks/mds-full}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'rhel_8}', 'tasks/mds-full}', 'whitelist_health', 'whitelist_wrongly_marked_down}']

Failure: "2021-07-02T13:25:39.248379+0000 osd.4 (osd.4) 8 : cluster [ERR] 5.0s0 deep-scrub : stat mismatch, got 0/20 objects, 0/0 clones, 0/20 dirty, 0/0 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 0/138185 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes." in cluster log
1 jobs: ['6249134']
suites: ['2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'fs/thrash/workloads/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/none', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}']

Dead: None
2 jobs: ['6249130', '6249146']

Failure: "2021-07-02T11:07:16.039174+0000 mds.l (mds.1) 8 : cluster [WRN] Scrub error on inode 0x10000000414 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-32/picture-13.jpg) see mds.l log and `damage ls` output for details" in cluster log
1 jobs: ['6249158']
suites: ['clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'distro/{rhel_8}', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'ranks/3', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/blogbench}', 'wsync/{yes}}']