2021-03-14T02:56:38.516 INFO:root:teuthology version: 1.1.0-b9656917 2021-03-14T02:56:38.519 INFO:teuthology.results:seed: '2236' 2021-03-14T02:59:57.126 INFO:root:teuthology version: 1.1.0-b9656917 2021-03-14T02:59:57.159 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-03-14T03:40:00.153 INFO:teuthology.results:Tests finished! gathering results... 2021-03-14T03:40:00.583 INFO:scrape:Found 3 jobs 2021-03-14T03:40:01.087 INFO:scrape:Found 1 distinct failure reasons 2021-03-14T03:40:01.088 INFO:scrape:Failure: "2021-03-14T03:19:44.046231+0000 mds.c (mds.0) 39 : cluster [WRN] Scrub error on inode 0x10000002867 (~mds0/stray5/10000002867) see mds.c log and `damage ls` output for details" in cluster log 2021-03-14T03:40:01.089 INFO:scrape:1 jobs: ['5963283'] 2021-03-14T03:40:01.089 INFO:scrape:suites: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{centos_8}', 'fs:workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'ranks/1', 'scrub/yes', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/blogbench}}'] 2021-03-14T03:40:01.090 INFO:scrape: 2021-03-14T03:40:01.091 INFO:teuthology.results:Sending results to mchangir@redhat.com: Test Run: mchangir-2021-03-14_02:55:52-fs:workload-wip-mchangir-mds-scrub-error-with-background-task-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/mchangir-2021-03-14_02:55:52-fs:workload-wip-mchangir-mds-scrub-error-with-background-task-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/mchangir-2021-03-14_02:55:52-fs:workload-wip-mchangir-mds-scrub-error-with-background-task-distro-basic-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 2 Fail ================================================================= [5963283] fs:workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} ----------------------------------------------------------------- time: 00:32:09 info: http://pulpito.ceph.com/mchangir-2021-03-14_02:55:52-fs:workload-wip-mchangir-mds-scrub-error-with-background-task-distro-basic-smithi/5963283/ log: http://qa-proxy.ceph.com/teuthology/mchangir-2021-03-14_02:55:52-fs:workload-wip-mchangir-mds-scrub-error-with-background-task-distro-basic-smithi/5963283/ "2021-03-14T03:19:44.046231+0000 mds.c (mds.0) 39 : cluster [WRN] Scrub error on inode 0x10000002867 (~mds0/stray5/10000002867) see mds.c log and `damage ls` output for details" in cluster log Pass ================================================================= [5963284] fs:workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} time: 00:36:07 info: http://pulpito.ceph.com/mchangir-2021-03-14_02:55:52-fs:workload-wip-mchangir-mds-scrub-error-with-background-task-distro-basic-smithi/5963284/ [5963285] fs:workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} time: 00:24:20 info: http://pulpito.ceph.com/mchangir-2021-03-14_02:55:52-fs:workload-wip-mchangir-mds-scrub-error-with-background-task-distro-basic-smithi/5963285/