2021-03-05T13:27:34.025 INFO:root:teuthology version: 1.1.0-3082387b 2021-03-05T13:27:34.029 INFO:teuthology.results:seed: '7021' 2021-03-05T13:29:33.025 INFO:root:teuthology version: 1.1.0-3082387b 2021-03-05T13:29:33.030 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-03-05T14:31:37.595 INFO:teuthology.results:Tests finished! gathering results... 2021-03-05T14:31:37.915 INFO:scrape:Found 3 jobs 2021-03-05T14:31:38.395 INFO:scrape:Found 1 distinct failure reasons 2021-03-05T14:31:38.396 INFO:scrape:Failure: "2021-03-05T13:50:16.778415+0000 mon.a (mon.0) 308 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log 2021-03-05T14:31:38.397 INFO:scrape:1 jobs: ['5939223'] 2021-03-05T14:31:38.397 INFO:scrape:suites: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{ubuntu_latest}', 'fs:workload/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'ranks/5', 'rhel_8}', 'scrub/yes', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/ffsb}}'] 2021-03-05T14:31:38.398 INFO:scrape: 2021-03-05T14:31:38.398 INFO:teuthology.results:Sending results to mchangir@redhat.com: Test Run: mchangir-2021-03-05_13:27:07-fs:workload-wip-mchangir-mds-scrub-error-with-background-task-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/mchangir-2021-03-05_13:27:07-fs:workload-wip-mchangir-mds-scrub-error-with-background-task-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/mchangir-2021-03-05_13:27:07-fs:workload-wip-mchangir-mds-scrub-error-with-background-task-distro-basic-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 2 Fail ================================================================= [5939223] fs:workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:55:43 info: http://pulpito.ceph.com/mchangir-2021-03-05_13:27:07-fs:workload-wip-mchangir-mds-scrub-error-with-background-task-distro-basic-smithi/5939223/ log: http://qa-proxy.ceph.com/teuthology/mchangir-2021-03-05_13:27:07-fs:workload-wip-mchangir-mds-scrub-error-with-background-task-distro-basic-smithi/5939223/ "2021-03-05T13:50:16.778415+0000 mon.a (mon.0) 308 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log Pass ================================================================= [5939221] fs:workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/fs/misc}} time: 00:44:36 info: http://pulpito.ceph.com/mchangir-2021-03-05_13:27:07-fs:workload-wip-mchangir-mds-scrub-error-with-background-task-distro-basic-smithi/5939221/ [5939222] fs:workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} time: 00:31:55 info: http://pulpito.ceph.com/mchangir-2021-03-05_13:27:07-fs:workload-wip-mchangir-mds-scrub-error-with-background-task-distro-basic-smithi/5939222/