2021-03-07T00:33:00.061 INFO:root:teuthology version: 1.1.0-3082387b 2021-03-07T00:33:00.063 INFO:teuthology.results:subset: '19/10' 2021-03-07T00:33:00.063 INFO:teuthology.results:seed: '4666' 2021-03-07T02:51:56.020 INFO:root:teuthology version: 1.1.0-3082387b 2021-03-07T02:51:56.055 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-03-07T04:24:03.233 INFO:teuthology.results:Tests finished! gathering results... 2021-03-07T04:24:11.455 INFO:scrape:Found 60 jobs 2021-03-07T04:24:13.086 INFO:scrape:Found 2 distinct failure reasons 2021-03-07T04:24:13.087 INFO:scrape:Crash: Command failed on smithi186 with status 1: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp' 2021-03-07T04:24:13.088 INFO:scrape:ceph version 14.2.16-432-g9b82969 (9b829698f3d6269668c9402131bbfba81df6df0f) nautilus (stable) 1: (()+0xf630) [0x7f8b6c1a7630] 2: (gsignal()+0x37) [0x7f8b6ad84387] 3: (abort()+0x148) [0x7f8b6ad85a78] 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x199) [0x7f8b6e2e9e46] 5: (()+0x25cfbf) [0x7f8b6e2e9fbf] 6: (ProtocolV1::handle_message_footer(char*, int)+0xf3d) [0x7f8b6e5d82ed] 7: (()+0x543aed) [0x7f8b6e5d0aed] 8: (AsyncConnection::process()+0x186) [0x7f8b6e5c26f6] 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa15) [0x7f8b6e611d65] 10: (()+0x58b6c7) [0x7f8b6e6186c7] 11: (()+0x82a94f) [0x7f8b6e8b794f] 12: (()+0x7ea5) [0x7f8b6c19fea5] 13: (clone()+0x6d) [0x7f8b6ae4c96d] 2021-03-07T04:24:13.088 INFO:scrape:2 jobs: ['5940682', '5940703'] 2021-03-07T04:24:13.089 INFO:scrape:suites intersection: ['begin', 'ceph-thrash/default', 'clusters/9-mds-3-standby', 'conf/{client', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multimds/thrash/{0-supported-random-distro$/{centos_latest}', 'osd}', 'overrides/{distro/rhel/{k-distro', 'overrides/{fuse-default-perm-no', 'rhel_latest}', 'session_timeout', 'tasks/cfuse_workunit_suites_fsstress}', 'thrash/{frag_enable', 'thrash_debug}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-07T04:24:13.089 INFO:scrape:suites union: ['begin', 'ceph-thrash/default', 'clusters/9-mds-3-standby', 'conf/{client', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multimds/thrash/{0-supported-random-distro$/{centos_latest}', 'objectstore-ec/bluestore-comp', 'objectstore-ec/filestore-xfs', 'osd}', 'overrides/{distro/rhel/{k-distro', 'overrides/{fuse-default-perm-no', 'rhel_latest}', 'session_timeout', 'tasks/cfuse_workunit_suites_fsstress}', 'thrash/{frag_enable', 'thrash_debug}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-07T04:24:13.090 INFO:scrape: 2021-03-07T04:24:13.090 INFO:scrape:Failure: Command failed (workunit test suites/fsstress.sh) on smithi112 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9b829698f3d6269668c9402131bbfba81df6df0f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' 2021-03-07T04:24:13.091 INFO:scrape:1 jobs: ['5940763'] 2021-03-07T04:24:13.091 INFO:scrape:suites: ['begin', 'ceph-thrash/default', 'clusters/9-mds-3-standby', 'conf/{client', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multimds/thrash/{0-supported-random-distro$/{ubuntu_latest}', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{distro/rhel/{k-distro', 'overrides/{fuse-default-perm-no', 'rhel_latest}', 'session_timeout', 'tasks/cfuse_workunit_suites_fsstress}', 'thrash/{frag_enable', 'thrash_debug}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-07T04:24:13.091 INFO:scrape: 2021-03-07T04:24:13.095 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/ failed: 3 dead: 0 running: 0 waiting: 0 queued: 0 passed: 57 Fail ================================================================= [5940682] multimds/thrash/{0-supported-random-distro$/{centos_latest} begin ceph-thrash/default clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress} ----------------------------------------------------------------- time: 00:23:37 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940682/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940682/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1fec910be1664f0d88d536300cbe7782 Command failed on smithi186 with status 1: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp' [5940703] multimds/thrash/{0-supported-random-distro$/{centos_latest} begin ceph-thrash/default clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress} ----------------------------------------------------------------- time: 00:19:57 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940703/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940703/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8ca10f1786cc43578694f7f9c1e27db5 Command failed (workunit test suites/fsstress.sh) on smithi201 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9b829698f3d6269668c9402131bbfba81df6df0f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' [5940763] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/default clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress} ----------------------------------------------------------------- time: 00:15:36 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940763/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940763/ sentry: https://sentry.ceph.com/organizations/ceph/?query=200714f33eda4b659334cfc0450257dc Command failed (workunit test suites/fsstress.sh) on smithi112 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9b829698f3d6269668c9402131bbfba81df6df0f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' Pass ================================================================= [5940680] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports} time: 00:16:56 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940680/ [5940684] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep} time: 00:17:45 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940684/ [5940686] multimds/basic/{0-supported-random-distro$/{rhel_7} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots} time: 00:24:55 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940686/ [5940688] multimds/basic/{0-supported-random-distro$/{ubuntu_16.04} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build} time: 01:55:17 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940688/ [5940689] multimds/basic/{0-supported-random-distro$/{ubuntu_16.04} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc} time: 00:35:09 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940689/ [5940690] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats} time: 00:18:02 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940690/ [5940691] multimds/basic/{0-supported-random-distro$/{ubuntu_16.04} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench} time: 00:27:49 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940691/ [5940692] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/default clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd} time: 00:09:56 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940692/ [5940693] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind} time: 00:53:21 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940693/ [5940694] multimds/basic/{0-supported-random-distro$/{rhel_7} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench} time: 01:01:59 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940694/ [5940695] multimds/basic/{0-supported-random-distro$/{ubuntu_16.04} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb} time: 01:48:30 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940695/ [5940696] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress} time: 00:19:25 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940696/ [5940697] multimds/basic/{0-supported-random-distro$/{rhel_7} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx} time: 00:20:15 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940697/ [5940700] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd} time: 00:11:01 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940700/ [5940701] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports} time: 00:14:50 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940701/ [5940706] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep} time: 00:16:40 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940706/ [5940707] multimds/basic/{0-supported-random-distro$/{ubuntu_16.04} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots} time: 00:30:08 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940707/ [5940709] multimds/basic/{0-supported-random-distro$/{rhel_7} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build} time: 00:33:51 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940709/ [5940711] multimds/basic/{0-supported-random-distro$/{ubuntu_16.04} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc} time: 00:39:14 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940711/ [5940713] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats} time: 00:14:44 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940713/ [5940715] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench} time: 00:25:19 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940715/ [5940717] multimds/thrash/{0-supported-random-distro$/{ubuntu_16.04} begin ceph-thrash/default clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd} time: 00:10:58 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940717/ [5940719] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind} time: 00:53:30 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940719/ [5940721] multimds/basic/{0-supported-random-distro$/{ubuntu_16.04} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench} time: 01:02:03 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940721/ [5940723] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb} time: 00:32:37 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940723/ [5940725] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress} time: 00:15:09 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940725/ [5940727] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx} time: 00:40:09 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940727/ [5940729] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd} time: 00:12:34 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940729/ [5940731] multimds/basic/{0-supported-random-distro$/{ubuntu_16.04} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports} time: 00:12:43 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940731/ [5940733] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/default clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress} time: 00:20:52 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940733/ [5940735] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep} time: 00:20:45 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940735/ [5940737] multimds/basic/{0-supported-random-distro$/{ubuntu_16.04} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots} time: 00:24:21 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940737/ [5940739] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build} time: 02:15:34 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940739/ [5940741] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc} time: 00:46:20 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940741/ [5940743] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats} time: 00:14:51 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940743/ [5940745] multimds/basic/{0-supported-random-distro$/{ubuntu_16.04} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench} time: 00:31:42 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940745/ [5940747] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/default clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd} time: 00:09:34 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940747/ [5940749] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind} time: 00:51:51 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940749/ [5940751] multimds/basic/{0-supported-random-distro$/{rhel_7} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench} time: 01:01:21 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940751/ [5940753] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb} time: 00:32:35 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940753/ [5940755] multimds/basic/{0-supported-random-distro$/{rhel_7} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress} time: 00:20:47 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940755/ [5940757] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx} time: 00:19:43 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940757/ [5940759] multimds/basic/{0-supported-random-distro$/{rhel_7} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd} time: 00:13:20 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940759/ [5940761] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports} time: 00:13:48 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940761/ [5940765] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep} time: 00:20:17 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940765/ [5940767] multimds/basic/{0-supported-random-distro$/{rhel_7} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots} time: 00:31:46 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940767/ [5940769] multimds/basic/{0-supported-random-distro$/{rhel_7} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build} time: 00:44:08 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940769/ [5940772] multimds/basic/{0-supported-random-distro$/{rhel_7} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc} time: 01:02:02 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940772/ [5940773] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats} time: 00:14:47 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940773/ [5940776] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench} time: 00:20:09 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940776/ [5940778] multimds/thrash/{0-supported-random-distro$/{rhel_7} begin ceph-thrash/default clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd} time: 00:13:02 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940778/ [5940780] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind} time: 00:51:39 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940780/ [5940782] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench} time: 01:00:41 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940782/ [5940784] multimds/basic/{0-supported-random-distro$/{ubuntu_16.04} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb} time: 00:41:24 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940784/ [5940786] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress} time: 00:17:33 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940786/ [5940788] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx} time: 00:36:34 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940788/ [5940790] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd} time: 00:13:06 info: http://pulpito.ceph.com/yuriw-2021-03-06_03:17:02-multimds-nautilus-distro-basic-smithi/5940790/