2021-11-03T17:47:23.301 INFO:root:teuthology version: 1.1.0-c56135d1 2021-11-03T17:47:23.303 INFO:teuthology.results:subset: '555/9999' 2021-11-03T17:47:23.303 INFO:teuthology.results:seed: '7827' 2021-11-03T23:30:50.974 INFO:root:teuthology version: 1.1.0-c56135d1 2021-11-03T23:30:51.003 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-11-04T03:12:07.827 INFO:teuthology.results:Tests finished! gathering results... 2021-11-04T03:12:10.388 INFO:scrape:Found 24 jobs 2021-11-04T03:12:12.467 INFO:scrape:Found 1 distinct failure reasons 2021-11-04T03:12:12.468 INFO:scrape:Failure: The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} 2021-11-04T03:12:12.468 INFO:scrape:1 jobs: ['6468796'] 2021-11-04T03:12:12.469 INFO:scrape:suites: ['begin', 'clusters/9-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/no', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'multimds/basic/{0-supported-random-distro$/{ubuntu_18.04}', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/testing/{flavor/centos_latest', 'q_check_counter/check_counter', 'session_timeout', 'tasks/cephfs_test_exports}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-04T03:12:12.469 INFO:scrape: 2021-11-04T03:12:12.471 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba ================================================================= info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 23 Fail ================================================================= [6468796] multimds/basic/{0-supported-random-distro$/{ubuntu_18.04} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports} ----------------------------------------------------------------- time: 00:36:13 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468796/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468796/ The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} Pass ================================================================= [6468781] multimds/basic/{0-supported-random-distro$/{ubuntu_18.04} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb} time: 00:34:28 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468781/ [6468783] multimds/thrash/{0-supported-random-distro$/{ubuntu_18.04} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress} time: 00:22:11 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468783/ [6468785] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress} time: 00:11:01 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468785/ [6468787] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep} time: 00:56:57 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468787/ [6468788] multimds/basic/{0-supported-random-distro$/{ubuntu_18.04} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx} time: 00:37:30 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468788/ [6468790] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd} time: 00:12:04 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468790/ [6468792] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd} time: 00:11:51 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468792/ [6468794] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind} time: 02:00:54 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468794/ [6468798] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress} time: 00:21:09 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468798/ [6468800] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep} time: 00:54:24 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468800/ [6468801] multimds/basic/{0-supported-random-distro$/{ubuntu_18.04} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots} time: 00:23:15 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468801/ [6468803] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build} time: 01:28:59 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468803/ [6468805] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd} time: 00:13:49 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468805/ [6468807] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind} time: 01:43:47 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468807/ [6468809] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc} time: 00:28:12 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468809/ [6468810] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats} time: 00:12:58 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468810/ [6468812] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress} time: 00:17:12 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468812/ [6468814] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep} time: 01:06:46 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468814/ [6468816] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench} time: 00:32:05 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468816/ [6468818] multimds/basic/{0-supported-random-distro$/{ubuntu_18.04} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench} time: 00:54:37 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468818/ [6468820] multimds/thrash/{0-supported-random-distro$/{ubuntu_18.04} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd} time: 00:11:44 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468820/ [6468822] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind} time: 02:13:24 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468822/ [6468823] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb} time: 00:27:01 info: http://pulpito.ceph.com/teuthology-2021-10-30_13:15:03-multimds-octopus-distro-default-gibba/6468823/