2020-09-17T11:58:11.600 INFO:root:teuthology version: 1.0.0-55bd2dc
2020-09-17T11:58:11.602 INFO:teuthology.results:subset: '0/4'
2020-09-17T11:58:11.602 INFO:teuthology.results:seed: '291'
2020-09-17T17:14:52.084 INFO:root:teuthology version: 1.0.0-55bd2dc
2020-09-17T17:14:52.109 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-09-18T03:25:12.126 INFO:teuthology.results:Tests finished! gathering results...
2020-09-18T03:25:51.661 INFO:scrape:Found 320 jobs
2020-09-18T03:26:16.296 INFO:scrape:Found 12 distinct failure reasons
2020-09-18T03:26:16.298 INFO:scrape:Failure: Test failure: test_ephemeral_pin_dist_conf_off_merge (tasks.cephfs.test_exports.TestEphemeralPins)
2020-09-18T03:26:16.298 INFO:scrape:20 jobs: ['5442614', '5442598', '5442502', '5442326', '5442374', '5442454', '5442630', '5442390', '5442518', '5442406', '5442550', '5442582', '5442470', '5442438', '5442486', '5442358', '5442534', '5442342', '5442566', '5442422']
2020-09-18T03:26:16.299 INFO:scrape:suites intersection: ['begin', 'conf/{client', 'fuse-default-perm-no}', 'mds', 'mon', 'osd}', 'overrides/{basic/{frag_enable', 'q_check_counter/check_counter', 'tasks/cephfs_test_exports}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-09-18T03:26:16.299 INFO:scrape:suites union: ['begin', 'clusters/3-mds', 'clusters/9-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/no', 'inline/yes', 'k-testing}', 'mds', 'mon', 'mount/fuse', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'multimds/basic/{0-supported-random-distro$/{centos_8}', 'multimds/basic/{0-supported-random-distro$/{rhel_8}', 'multimds/basic/{0-supported-random-distro$/{ubuntu_latest}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore-ec/filestore-xfs', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'q_check_counter/check_counter', 'rhel_8}', 'tasks/cephfs_test_exports}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-09-18T03:26:16.300 INFO:scrape:
2020-09-18T03:26:16.300 INFO:scrape:Failure: Test failure: test_multimds_hardlink (tasks.cephfs.test_snapshots.TestSnapshots)
2020-09-18T03:26:16.300 INFO:scrape:15 jobs: ['5442505', '5442617', '5442425', '5442457', '5442489', '5442633', '5442377', '5442441', '5442393', '5442585', '5442569', '5442361', '5442521', '5442329', '5442553']
2020-09-18T03:26:16.301 INFO:scrape:suites intersection: ['begin', 'conf/{client', 'fuse-default-perm-no}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'osd}', 'overrides/{basic/{frag_enable', 'q_check_counter/check_counter', 'tasks/cephfs_test_snapshots}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-09-18T03:26:16.301 INFO:scrape:suites union: ['begin', 'clusters/3-mds', 'clusters/9-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/no', 'inline/yes', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'multimds/basic/{0-supported-random-distro$/{centos_8}', 'multimds/basic/{0-supported-random-distro$/{rhel_8}', 'multimds/basic/{0-supported-random-distro$/{ubuntu_latest}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore-ec/filestore-xfs', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'q_check_counter/check_counter', 'rhel_8}', 'tasks/cephfs_test_snapshots}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-09-18T03:26:16.302 INFO:scrape:
2020-09-18T03:26:16.302 INFO:scrape:Failure: Test failure: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots)
2020-09-18T03:26:16.302 INFO:scrape:5 jobs: ['5442601', '5442409', '5442473', '5442537', '5442345']
2020-09-18T03:26:16.303 INFO:scrape:suites intersection: ['begin', 'clusters/3-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/no', 'mds', 'mon', 'mount/fuse', 'osd}', 'overrides/{basic/{frag_enable', 'q_check_counter/check_counter', 'tasks/cephfs_test_snapshots}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-09-18T03:26:16.303 INFO:scrape:suites union: ['begin', 'clusters/3-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/no', 'mds', 'mon', 'mount/fuse', 'multimds/basic/{0-supported-random-distro$/{centos_8}', 'multimds/basic/{0-supported-random-distro$/{rhel_8}', 'multimds/basic/{0-supported-random-distro$/{ubuntu_latest}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore-ec/filestore-xfs', 'osd}', 'overrides/{basic/{frag_enable', 'q_check_counter/check_counter', 'tasks/cephfs_test_snapshots}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-09-18T03:26:16.304 INFO:scrape:
2020-09-18T03:26:16.304 INFO:scrape:Failure: "2020-09-17T14:58:52.791648+0000 mds.e (mds.0) 1 : cluster [WRN] evicting unresponsive client smithi039: (4580), after waiting 49.5299 seconds during MDS startup" in cluster log
2020-09-18T03:26:16.304 INFO:scrape:3 jobs: ['5442479', '5442440', '5442433']
2020-09-18T03:26:16.306 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'whitelist_health']
2020-09-18T03:26:16.306 INFO:scrape:suites union: ['begin', 'centos_latest', 'ceph-thrash/mds', 'clusters/3-mds-2-standby', 'clusters/9-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/yes', 'k-testing}', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'mount/kclient/{mount', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'msgr-failures/none', 'multimds/basic/{0-supported-random-distro$/{ubuntu_latest}', 'multimds/thrash/{0-supported-random-distro$/{rhel_8}', 'multimds/verify/{begin', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{fuse-default-perm-no', 'q_check_counter/check_counter', 'rhel_8}', 'tasks/cfuse_workunit_suites_ffsb}', 'tasks/cfuse_workunit_suites_fsstress', 'tasks/cfuse_workunit_suites_pjd}', 'thrash/{frag_enable', 'thrash_debug}', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'whitelist_wrongly_marked_down}}']
2020-09-18T03:26:16.306 INFO:scrape:
2020-09-18T03:26:16.307 INFO:scrape:Dead: b'2020-09-18T02:53:02.014 DEBUG:teuthology.exit:Finished running handlers'
2020-09-18T03:26:16.307 INFO:scrape:2 jobs: ['5442511', '5442536']
2020-09-18T03:26:16.308 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'overrides/{fuse-default-perm-no', 'whitelist_health']
2020-09-18T03:26:16.309 INFO:scrape:suites union: ['begin', 'centos_latest', 'ceph-thrash/mds', 'clusters/3-mds-2-standby', 'clusters/9-mds', 'conf/{client', 'k-testing}', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'mount/kclient/{mount', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multimds/thrash/{0-supported-random-distro$/{centos_8}', 'multimds/verify/{begin', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{fuse-default-perm-no', 'tasks/cfuse_workunit_suites_dbench', 'tasks/cfuse_workunit_suites_fsstress}', 'thrash/{frag_enable', 'thrash_debug}', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'whitelist_wrongly_marked_down}}']
2020-09-18T03:26:16.309 INFO:scrape:
2020-09-18T03:26:16.310 INFO:scrape:Failure: Command failed on smithi201 with status 1: 'sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel'
2020-09-18T03:26:16.310 INFO:scrape:4 jobs: ['5442508', '5442520', '5442429', '5442343']
2020-09-18T03:26:16.311 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'whitelist_health']
2020-09-18T03:26:16.311 INFO:scrape:suites union: ['begin', 'centos_latest', 'ceph-thrash/mds', 'clusters/3-mds', 'clusters/3-mds-2-standby', 'conf/{client', 'fuse-default-perm-no}', 'inline/no', 'k-testing}', 'mds', 'mon', 'mon-debug', 'mount/fuse', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/none', 'multimds/basic/{0-supported-random-distro$/{centos_8}', 'multimds/basic/{0-supported-random-distro$/{ubuntu_latest}', 'multimds/thrash/{0-supported-random-distro$/{centos_8}', 'multimds/verify/{begin', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/filestore-xfs', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{fuse-default-perm-no', 'q_check_counter/check_counter', 'tasks/cfuse_workunit_misc}', 'tasks/cfuse_workunit_norstats}', 'tasks/cfuse_workunit_suites_fsstress', 'tasks/cfuse_workunit_suites_fsstress}', 'thrash/{frag_enable', 'thrash_debug}', 'validater/lockdep}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'whitelist_wrongly_marked_down}}']
2020-09-18T03:26:16.311 INFO:scrape:
2020-09-18T03:26:16.312 INFO:scrape:Failure: {'Failure object was': {'smithi025.front.sepia.ceph.com': {'msg': 'Failed to download packages: Cannot download Packages/boost-random-1.66.0-6.el8.x86_64.rpm: All mirrors were tried', 'results': [], 'invocation': {'module_args': {'name': ['dnf-utils', 'sysstat', 'libedit', 'boost-thread', 'xfsprogs', 'gdisk', 'parted', 'libgcrypt', 'fuse-libs', 'openssl', 'libuuid', 'podman', 'podman-docker', 'attr', 'ant', 'lsof', 'gettext', 'bc', 'xfsdump', 'blktrace', 'usbredir', 'libev-devel', 'valgrind', 'nfs-utils', 'ncurses-devel', 'gcc', 'git', 'python3-nose', 'python3-virtualenv', 'genisoimage', 'qemu-img', 'qemu-kvm-core', 'qemu-kvm-block-rbd', 'libacl-devel', 'dbench', 'autoconf', 'gdb', 'iozone'], 'state': 'present', 'allow_downgrade': False, 'autoremove': False, 'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [], 'disablerepo': [], 'download_only': False, 'enable_plugin': [], 'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery': True, 'install_weak_deps': True, 'security': False, 'skip_broken': False, 'update_cache': False, 'update_only': False, 'validate_certs': True, 'lock_timeout': 30, 'conf_file': 'None', 'disable_excludes': 'None', 'download_dir': 'None', 'list': 'None', 'releasever': 'None'}}, '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', '/')"}
2020-09-18T03:26:16.312 INFO:scrape:1 jobs: ['5442597']
2020-09-18T03:26:16.313 INFO:scrape:suites: ['begin', 'clusters/3-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/no', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'multimds/basic/{0-supported-random-distro$/{centos_8}', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/testing/{flavor/centos_latest', 'q_check_counter/check_counter', 'tasks/cfuse_workunit_suites_pjd}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-09-18T03:26:16.313 INFO:scrape:
2020-09-18T03:26:16.313 INFO:scrape:Failure: "2020-09-17T15:06:29.763107+0000 mon.b (mon.0) 233 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log
2020-09-18T03:26:16.313 INFO:scrape:1 jobs: ['5442484']
2020-09-18T03:26:16.314 INFO:scrape:suites: ['begin', 'clusters/3-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/no', 'mds', 'mon', 'mount/fuse', 'multimds/basic/{0-supported-random-distro$/{ubuntu_latest}', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{basic/{frag_enable', 'q_check_counter/check_counter', 'tasks/cfuse_workunit_suites_fsx}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-09-18T03:26:16.314 INFO:scrape:
2020-09-18T03:26:16.314 INFO:scrape:Failure: reached maximum tries (50) after waiting for 300 seconds
2020-09-18T03:26:16.315 INFO:scrape:1 jobs: ['5442600']
2020-09-18T03:26:16.315 INFO:scrape:suites: ['centos_latest', 'clusters/9-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'multimds/verify/{begin', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{fuse-default-perm-no', 'tasks/cfuse_workunit_suites_dbench', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}']
2020-09-18T03:26:16.315 INFO:scrape:
2020-09-18T03:26:16.316 INFO:scrape:Failure: 
2020-09-18T03:26:16.316 INFO:scrape:1 jobs: ['5442376']
2020-09-18T03:26:16.316 INFO:scrape:suites: ['centos_latest', 'clusters/9-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'multimds/verify/{begin', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{fuse-default-perm-no', 'tasks/cfuse_workunit_suites_fsstress', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}']
2020-09-18T03:26:16.317 INFO:scrape:
2020-09-18T03:26:16.317 INFO:scrape:Failure: "2020-09-17T15:39:14.491316+0000 mds.c (mds.0) 1 : cluster [WRN] client.4707 isn't responding to mclientcaps(revoke), ino 0x30000000258 pending pAsLsXsFscr issued pAsLsXsFsxcrwb, sent 64.920503 seconds ago" in cluster log
2020-09-18T03:26:16.317 INFO:scrape:2 jobs: ['5442513', '5442414']
2020-09-18T03:26:16.318 INFO:scrape:suites intersection: ['begin', 'clusters/3-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/no', 'mds', 'mon', 'mount/fuse', 'multimds/basic/{0-supported-random-distro$/{rhel_8}', 'osd}', 'overrides/{basic/{frag_enable', 'q_check_counter/check_counter', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-09-18T03:26:16.318 INFO:scrape:suites union: ['begin', 'clusters/3-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/no', 'mds', 'mon', 'mount/fuse', 'multimds/basic/{0-supported-random-distro$/{rhel_8}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{basic/{frag_enable', 'q_check_counter/check_counter', 'tasks/cfuse_workunit_suites_blogbench}', 'tasks/cfuse_workunit_suites_ffsb}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-09-18T03:26:16.318 INFO:scrape:
2020-09-18T03:26:16.319 INFO:scrape:Failure: Command failed (workunit test suites/fsstress.sh) on smithi105 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=363c17e70648676a18fb7610af1e34949b2feeac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh'
2020-09-18T03:26:16.319 INFO:scrape:3 jobs: ['5442495', '5442527', '5442344']
2020-09-18T03:26:16.319 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'overrides/{fuse-default-perm-no', 'whitelist_health']
2020-09-18T03:26:16.319 INFO:scrape:suites union: ['begin', 'centos_latest', 'ceph-thrash/mds', 'clusters/3-mds-2-standby', 'clusters/9-mds', 'conf/{client', 'k-testing}', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'mount/kclient/{mount', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multimds/thrash/{0-supported-random-distro$/{rhel_8}', 'multimds/thrash/{0-supported-random-distro$/{ubuntu_latest}', 'multimds/verify/{begin', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{fuse-default-perm-no', 'tasks/cfuse_workunit_suites_fsstress', 'tasks/cfuse_workunit_suites_fsstress}', 'thrash/{frag_enable', 'thrash_debug}', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'whitelist_wrongly_marked_down}}']
2020-09-18T03:26:16.320 INFO:scrape:
2020-09-18T03:26:16.337 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/
failed:  55
dead:    3
running: 0
waiting: 0
queued:  0
passed:  262


Fail
=================================================================
[5442326]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:09:21
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442326/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442326/
sentry: https://sentry.ceph.com/sentry/sepia/?query=2e6db40eda9c407990060a55ba06155d

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442329]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:13:21
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442329/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442329/
sentry: https://sentry.ceph.com/sentry/sepia/?query=1018796cbcf342e28fecd9fe322a5787

    Test failure: test_multimds_hardlink
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442342]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:11:03
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442342/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442342/
sentry: https://sentry.ceph.com/sentry/sepia/?query=ebdb50084d16436390cc0105eb7d1914

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442343]  multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
-----------------------------------------------------------------
time:   00:03:30
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442343/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442343/
sentry: https://sentry.ceph.com/sentry/sepia/?query=b78b9c00f46d457d91816217f42037e7

    Command failed on smithi166 with status 1: 'sudo yum -y install ceph-
    radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-
    mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-
    mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2
    librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-
    mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-
    devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-
    devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel
    libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel'

[5442344]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
-----------------------------------------------------------------
time:   00:43:13
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442344/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442344/
sentry: https://sentry.ceph.com/sentry/sepia/?query=f48c69fdad414ac1a5dc1e2e82071f42

    Command failed (workunit test suites/fsstress.sh) on smithi044 with status
    2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd --
    /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=363c17e70648676a18fb7610af1e34949b2feeac
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 6h
    /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'

[5442345]  multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:10:55
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442345/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442345/
sentry: https://sentry.ceph.com/sentry/sepia/?query=d9a073814fb7491a8ae9e2f97928895c

    Test failure: test_kill_mdstable
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442358]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:10:49
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442358/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442358/
sentry: https://sentry.ceph.com/sentry/sepia/?query=d85940ddc4e44e05be6a7120274f725a

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442361]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:12:01
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442361/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442361/
sentry: https://sentry.ceph.com/sentry/sepia/?query=e7091dbe259b4ab686f506529b1b1f05

    Test failure: test_multimds_hardlink
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442374]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:11:09
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442374/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442374/
sentry: https://sentry.ceph.com/sentry/sepia/?query=8b52312609e94cbc85fa49fa4cfa0d14

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442376]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
-----------------------------------------------------------------
time:   03:49:28
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442376/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442376/
sentry: https://sentry.ceph.com/sentry/sepia/?query=85d090d7cd794d49878afb0fec4e3abe

[5442377]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:12:29
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442377/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442377/
sentry: https://sentry.ceph.com/sentry/sepia/?query=8fd93111d7644480bff4ac447d9030f8

    Test failure: test_multimds_hardlink
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442390]  multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:11:14
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442390/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442390/
sentry: https://sentry.ceph.com/sentry/sepia/?query=9faf815f455447829eee62115c250cdd

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442393]  multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:13:23
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442393/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442393/
sentry: https://sentry.ceph.com/sentry/sepia/?query=fa5e3f7cfefa49a4b89bb06d10add9db

    Test failure: test_multimds_hardlink
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442406]  multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:11:01
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442406/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442406/
sentry: https://sentry.ceph.com/sentry/sepia/?query=6a92cf4b4d234608a73cf50a81baf246

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442409]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:08:25
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442409/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442409/
sentry: https://sentry.ceph.com/sentry/sepia/?query=25908dbf5028462e84cb366e81f48ef8

    Test failure: test_kill_mdstable
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442414]  multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
-----------------------------------------------------------------
time:   00:24:35
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442414/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442414/

    "2020-09-17T13:28:17.036460+0000 mds.a (mds.2) 1 : cluster [WRN]
    client.4662 isn't responding to mclientcaps(revoke), ino 0x300000003ab
    pending pAsLsXsFscr issued pAsLsXsFscr, sent 63.770691 seconds ago" in
    cluster log

[5442422]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:11:06
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442422/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442422/
sentry: https://sentry.ceph.com/sentry/sepia/?query=c1fee90ab0624baaaed5a6b1b0c15e12

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442425]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:11:51
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442425/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442425/
sentry: https://sentry.ceph.com/sentry/sepia/?query=5d7f2fbcb63b4ef4950232e7a14903b6

    Test failure: test_multimds_hardlink
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442429]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
-----------------------------------------------------------------
time:   00:05:03
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442429/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442429/
sentry: https://sentry.ceph.com/sentry/sepia/?query=de43b2b894794ad99cd9ccb9df9412a5

    Command failed on smithi068 with status 1: 'sudo yum -y install ceph-
    radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-
    mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-
    mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2
    librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-
    mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-
    devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-
    devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel
    libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel'

[5442433]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
-----------------------------------------------------------------
time:   02:33:41
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442433/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442433/

    "2020-09-17T14:18:50.841738+0000 mon.a (mon.0) 236 : cluster [WRN] Health
    check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in
    cluster log

[5442438]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:11:26
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442438/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442438/
sentry: https://sentry.ceph.com/sentry/sepia/?query=4cd3eb4156214f37aa0bfb7c159345c6

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442440]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
-----------------------------------------------------------------
time:   02:58:08
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442440/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442440/

    "2020-09-17T16:26:06.182204+0000 mds.i (mds.8) 1 : cluster [WRN] evicting
    unresponsive client smithi075:0 (4642), after 314.248 seconds" in cluster
    log

[5442441]  multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:12:38
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442441/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442441/
sentry: https://sentry.ceph.com/sentry/sepia/?query=333061f081264acfbab04970ecc81e4a

    Test failure: test_multimds_hardlink
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442454]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:09:41
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442454/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442454/
sentry: https://sentry.ceph.com/sentry/sepia/?query=d337ad3d174f46bcbfd2dc3540a09e67

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442457]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:12:09
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442457/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442457/
sentry: https://sentry.ceph.com/sentry/sepia/?query=4f72d07010a54cabad700811bb694c47

    Test failure: test_multimds_hardlink
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442470]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:10:38
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442470/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442470/
sentry: https://sentry.ceph.com/sentry/sepia/?query=b1355a4d0a6f4720a6f1837a379f103c

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442473]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:08:06
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442473/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442473/
sentry: https://sentry.ceph.com/sentry/sepia/?query=ebc3398ec1db4f4e99d1872f45f97ac5

    Test failure: test_kill_mdstable
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442479]  multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
-----------------------------------------------------------------
time:   00:15:14
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442479/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442479/

    "2020-09-17T14:58:52.791648+0000 mds.e (mds.0) 1 : cluster [WRN] evicting
    unresponsive client smithi039: (4580), after waiting 49.5299 seconds during
    MDS startup" in cluster log

[5442484]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
-----------------------------------------------------------------
time:   01:20:27
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442484/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442484/

    "2020-09-17T15:06:29.763107+0000 mon.b (mon.0) 233 : cluster [WRN] Health
    check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log

[5442486]  multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:10:57
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442486/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442486/
sentry: https://sentry.ceph.com/sentry/sepia/?query=91b7b0ce8cce49b5bc2ceae20f76a013

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442489]  multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:11:46
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442489/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442489/
sentry: https://sentry.ceph.com/sentry/sepia/?query=09001a9aa187465b9115dd5ef6b8175f

    Test failure: test_multimds_hardlink
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442495]  multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
-----------------------------------------------------------------
time:   02:41:16
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442495/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442495/
sentry: https://sentry.ceph.com/sentry/sepia/?query=0be5637606bd4271921717be81850afd

    Command failed (workunit test suites/fsstress.sh) on smithi105 with status
    2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=363c17e70648676a18fb7610af1e34949b2feeac
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 6h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh'

[5442502]  multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:11:31
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442502/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442502/
sentry: https://sentry.ceph.com/sentry/sepia/?query=3285e31807a943438bb50a42e51f7785

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442505]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:26:29
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442505/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442505/
sentry: https://sentry.ceph.com/sentry/sepia/?query=1f3078327f5c40939343878f48fd11d8

    Test failure: test_multimds_hardlink
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442508]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
-----------------------------------------------------------------
time:   00:06:21
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442508/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442508/
sentry: https://sentry.ceph.com/sentry/sepia/?query=0c47e886e8df4322b0fe83b18ad14d4f

    Command failed on smithi201 with status 1: 'sudo yum -y install ceph-
    radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-
    mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-
    mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2
    librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-
    mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-
    devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-
    devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel
    libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel'

[5442513]  multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
-----------------------------------------------------------------
time:   02:08:26
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442513/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442513/

    "2020-09-17T15:39:14.491316+0000 mds.c (mds.0) 1 : cluster [WRN]
    client.4707 isn't responding to mclientcaps(revoke), ino 0x30000000258
    pending pAsLsXsFscr issued pAsLsXsFsxcrwb, sent 64.920503 seconds ago" in
    cluster log

[5442518]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:10:01
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442518/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442518/
sentry: https://sentry.ceph.com/sentry/sepia/?query=5aa5d649b1694fe6b128995e942c493f

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442520]  multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
-----------------------------------------------------------------
time:   00:06:52
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442520/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442520/
sentry: https://sentry.ceph.com/sentry/sepia/?query=535f2ee694854799aef6e1e00c6287eb

    Command failed on smithi022 with status 1: 'sudo yum -y install ceph-
    radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-
    mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-
    mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2
    librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-
    mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-
    devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-
    devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel
    libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel
    openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel
    libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-
    devel libtool libuuid-devel xfsprogs-devel'

[5442521]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:13:09
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442521/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442521/
sentry: https://sentry.ceph.com/sentry/sepia/?query=7176d8cf8b21483286dbd796e9257860

    Test failure: test_multimds_hardlink
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442527]  multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
-----------------------------------------------------------------
time:   06:59:38
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442527/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442527/
sentry: https://sentry.ceph.com/sentry/sepia/?query=f756c7c668014c748e4ad48aa5d7d97c

    Command failed (workunit test suites/fsstress.sh) on smithi110 with status
    124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=363c17e70648676a18fb7610af1e34949b2feeac
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 6h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh'

[5442534]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:11:16
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442534/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442534/
sentry: https://sentry.ceph.com/sentry/sepia/?query=a49a49a3a63d4630ade40dc62bd7efe5

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442537]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:07:53
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442537/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442537/
sentry: https://sentry.ceph.com/sentry/sepia/?query=069f8da6b72740749bbeaa21414a4c33

    Test failure: test_kill_mdstable
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442550]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:10:39
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442550/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442550/
sentry: https://sentry.ceph.com/sentry/sepia/?query=b8ae7cec66a64a278646b47b639ee03c

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442553]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:12:05
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442553/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442553/
sentry: https://sentry.ceph.com/sentry/sepia/?query=6e6569943896478fa7d902f31197427e

    Test failure: test_multimds_hardlink
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442566]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:11:16
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442566/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442566/
sentry: https://sentry.ceph.com/sentry/sepia/?query=55cf64714b1a4706b32a2f6ca20bd336

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442569]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:12:32
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442569/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442569/
sentry: https://sentry.ceph.com/sentry/sepia/?query=69c94dcb18c8469c93c891826943f464

    Test failure: test_multimds_hardlink
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442582]  multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:11:31
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442582/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442582/
sentry: https://sentry.ceph.com/sentry/sepia/?query=c876b98765874067b3a34b7eff8d81ad

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442585]  multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:12:44
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442585/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442585/
sentry: https://sentry.ceph.com/sentry/sepia/?query=a106b95f58854b238a5f8f7488fa1bb8

    Test failure: test_multimds_hardlink
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442598]  multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:10:43
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442598/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442598/
sentry: https://sentry.ceph.com/sentry/sepia/?query=64726c924c084b67ad1290e367f64e72

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442600]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
-----------------------------------------------------------------
time:   00:59:13
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442600/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442600/

    reached maximum tries (50) after waiting for 300 seconds

[5442601]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:07:46
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442601/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442601/
sentry: https://sentry.ceph.com/sentry/sepia/?query=da70a78a800b43db8fd5699547a59b33

    Test failure: test_kill_mdstable
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442614]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:11:02
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442614/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442614/
sentry: https://sentry.ceph.com/sentry/sepia/?query=b374aa4b12b645aa9037aa6c8889731f

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442617]  multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:11:40
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442617/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442617/
sentry: https://sentry.ceph.com/sentry/sepia/?query=b9892b427d6a44bea7112f5329491ecd

    Test failure: test_multimds_hardlink
    (tasks.cephfs.test_snapshots.TestSnapshots)

[5442630]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:11:11
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442630/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442630/
sentry: https://sentry.ceph.com/sentry/sepia/?query=197c5f8bd65f4e6c92490b926f58cd59

    Test failure: test_ephemeral_pin_dist_conf_off_merge
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5442633]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:12:22
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442633/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442633/
sentry: https://sentry.ceph.com/sentry/sepia/?query=297db29887374d72a7f6fdf4361e31e4

    Test failure: test_multimds_hardlink
    (tasks.cephfs.test_snapshots.TestSnapshots)



Dead
=================================================================
[5442511]  multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442511/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442511/

[5442536]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442536/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442536/

[5442597]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
-----------------------------------------------------------------
time:   00:04:45
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442597/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442597/
sentry: https://sentry.ceph.com/sentry/sepia/?query=756911d412404378ab2199c5db216160

    {'Failure object was': {'smithi025.front.sepia.ceph.com': {'msg': 'Failed
    to download packages: Cannot download Packages/boost-
    random-1.66.0-6.el8.x86_64.rpm: All mirrors were tried', 'results': [],
    'invocation': {'module_args': {'name': ['dnf-utils', 'sysstat', 'libedit',
    'boost-thread', 'xfsprogs', 'gdisk', 'parted', 'libgcrypt', 'fuse-libs',
    'openssl', 'libuuid', 'podman', 'podman-docker', 'attr', 'ant', 'lsof',
    'gettext', 'bc', 'xfsdump', 'blktrace', 'usbredir', 'libev-devel',
    'valgrind', 'nfs-utils', 'ncurses-devel', 'gcc', 'git', 'python3-nose',
    'python3-virtualenv', 'genisoimage', 'qemu-img', 'qemu-kvm-core', 'qemu-
    kvm-block-rbd', 'libacl-devel', 'dbench', 'autoconf', 'gdb', 'iozone'],
    'state': 'present', 'allow_downgrade': False, 'autoremove': False,
    'bugfix': False, 'disable_gpg_check': False, 'disable_plugin': [],
    'disablerepo': [], 'download_only': False, 'enable_plugin': [],
    'enablerepo': [], 'exclude': [], 'installroot': '/', 'install_repoquery':
    True, 'install_weak_deps': True, 'security': False, 'skip_broken': False,
    'update_cache': False, 'update_only': False, 'validate_certs': True,
    'lock_timeout': 30, 'conf_file': 'None', 'disable_excludes': 'None',
    'download_dir': 'None', 'list': 'None', 'releasever': 'None'}},
    '_ansible_no_log': False, 'changed': False}}, 'Traceback (most recent call
    last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-
    ansible_master/callback_plugins/failure_log.py", line 44, in log_failure
    log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com
    _git_teuthology_master/virtualenv/lib/python3.6/site-
    packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data],
    stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com
    _git_teuthology_master/virtualenv/lib/python3.6/site-
    packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data)
    File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/l
    ib/python3.6/site-packages/yaml/representer.py", line 27, in represent node
    = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_te
    uthology_master/virtualenv/lib/python3.6/site-
    packages/yaml/representer.py", line 48, in represent_data node =
    self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s
    rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-
    packages/yaml/representer.py", line 207, in represent_dict return
    self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo
    rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-
    packages/yaml/representer.py", line 118, in represent_mapping node_value =
    self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi
    t_teuthology_master/virtualenv/lib/python3.6/site-
    packages/yaml/representer.py", line 48, in represent_data node =
    self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s
    rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-
    packages/yaml/representer.py", line 207, in represent_dict return
    self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo
    rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-
    packages/yaml/representer.py", line 118, in represent_mapping node_value =
    self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi
    t_teuthology_master/virtualenv/lib/python3.6/site-
    packages/yaml/representer.py", line 48, in represent_data node =
    self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s
    rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-
    packages/yaml/representer.py", line 207, in represent_dict return
    self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo
    rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-
    packages/yaml/representer.py", line 118, in represent_mapping node_value =
    self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi
    t_teuthology_master/virtualenv/lib/python3.6/site-
    packages/yaml/representer.py", line 48, in represent_data node =
    self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s
    rc/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-
    packages/yaml/representer.py", line 207, in represent_dict return
    self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthwo
    rker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-
    packages/yaml/representer.py", line 118, in represent_mapping node_value =
    self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi
    t_teuthology_master/virtualenv/lib/python3.6/site-
    packages/yaml/representer.py", line 58, in represent_data node =
    self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce
    ph.com_git_teuthology_master/virtualenv/lib/python3.6/site-
    packages/yaml/representer.py", line 231, in represent_undefined raise
    RepresenterError("cannot represent an object", data)',
    'yaml.representer.RepresenterError': "('cannot represent an object', '/')"}



Pass
=================================================================
[5442327] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:14:35
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442327/

[5442328] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:30:30
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442328/

[5442330] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:27:59
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442330/

[5442331] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:31
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442331/

[5442332] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:26:56
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442332/

[5442333] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:17:46
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442333/

[5442334] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:49:00
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442334/

[5442335] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:14:21
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442335/

[5442336] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:41:17
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442336/

[5442337] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:39:12
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442337/

[5442338] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:19:13
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442338/

[5442339] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:14:12
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442339/

[5442340] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:18:47
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442340/

[5442341] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:13:38
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442341/

[5442346] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:35:59
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442346/

[5442347] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:33
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442347/

[5442348] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   00:41:57
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442348/

[5442349] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:13:51
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442349/

[5442350] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:29:58
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442350/

[5442351] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:16:34
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442351/

[5442352] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:20:46
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442352/

[5442353] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   02:17:49
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442353/

[5442354] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:14:57
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442354/

[5442355] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:14:17
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442355/

[5442356] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:34:26
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442356/

[5442357] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:12:33
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442357/

[5442359] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:16:57
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442359/

[5442360] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:21:51
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442360/

[5442362] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   02:14:42
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442362/

[5442363] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:11:52
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442363/

[5442364] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:23:00
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442364/

[5442365] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:15:50
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442365/

[5442366] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:43:00
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442366/

[5442367] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:14:23
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442367/

[5442368] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:38:02
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442368/

[5442369] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:31:52
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442369/

[5442370] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:15:41
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442370/

[5442371] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:39
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442371/

[5442372] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:53:29
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442372/

[5442373] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:10:42
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442373/

[5442375] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:16:00
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442375/

[5442378] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:14:12
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442378/

[5442379] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:40
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442379/

[5442380] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   02:54:30
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442380/

[5442381] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:14:38
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442381/

[5442382] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:59:02
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442382/

[5442383] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:14:01
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442383/

[5442384] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   00:50:27
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442384/

[5442385] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:39:43
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442385/

[5442386] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:22:01
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442386/

[5442387] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:14:06
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442387/

[5442388] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:23:11
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442388/

[5442389] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:29:37
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442389/

[5442391] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:15:00
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442391/

[5442392] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   00:56:55
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442392/

[5442394] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   02:51:17
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442394/

[5442395] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:37
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442395/

[5442396] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:00:42
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442396/

[5442397] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:20:53
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442397/

[5442398] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:32:03
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442398/

[5442399] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:15:26
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442399/

[5442400] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:35:36
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442400/

[5442401] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   02:07:32
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442401/

[5442402] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:14:57
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442402/

[5442403] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:14:14
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442403/

[5442404] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:40:23
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442404/

[5442405] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:13:21
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442405/

[5442407] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:10:41
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442407/

[5442408] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
time:   02:29:53
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442408/

[5442410] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:02:09
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442410/

[5442411] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:21:37
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442411/

[5442412] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:52:49
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442412/

[5442413] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:13:26
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442413/

[5442415] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:19
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442415/

[5442416] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:24:03
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442416/

[5442417] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:46:37
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442417/

[5442418] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:19:29
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442418/

[5442419] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:20:32
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442419/

[5442420] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:24:10
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442420/

[5442421] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:12:17
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442421/

[5442423] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:14
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442423/

[5442424] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:55:32
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442424/

[5442426] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   03:14:49
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442426/

[5442427] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:19:10
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442427/

[5442428] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   00:49:19
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442428/

[5442430] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:59:47
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442430/

[5442431] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:25
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442431/

[5442432] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:46:18
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442432/

[5442434] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:16:31
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442434/

[5442435] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:18:12
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442435/

[5442436] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:25:52
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442436/

[5442437] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:10:54
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442437/

[5442439] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:09:49
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442439/

[5442442] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   00:41:01
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442442/

[5442443] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:19:35
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442443/

[5442444] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   02:17:31
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442444/

[5442445] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:16:15
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442445/

[5442446] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:41:31
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442446/

[5442447] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:12
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442447/

[5442448] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:12:40
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442448/

[5442449] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:27:44
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442449/

[5442450] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:17:44
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442450/

[5442451] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:19:16
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442451/

[5442452] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:54:58
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442452/

[5442453] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:35:05
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442453/

[5442455] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:09:53
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442455/

[5442456] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:29:32
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442456/

[5442458] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:39:49
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442458/

[5442459] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:19:46
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442459/

[5442460] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:50:20
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442460/

[5442461] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:21:03
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442461/

[5442462] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   01:00:56
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442462/

[5442463] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:04
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442463/

[5442464] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   00:54:06
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442464/

[5442465] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:27:10
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442465/

[5442466] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:20:56
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442466/

[5442467] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:18:23
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442467/

[5442468] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:24:15
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442468/

[5442469] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:13:22
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442469/

[5442471] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:10:46
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442471/

[5442472] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
time:   02:03:15
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442472/

[5442474] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:44:51
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442474/

[5442475] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:19:52
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442475/

[5442476] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:11:19
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442476/

[5442477] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:16:58
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442477/

[5442478] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:22:57
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442478/

[5442480] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:22:19
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442480/

[5442481] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   01:53:57
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442481/

[5442482] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:16:48
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442482/

[5442483] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:16:52
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442483/

[5442485] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:12:08
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442485/

[5442487] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:17:23
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442487/

[5442488] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
time:   00:13:40
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442488/

[5442490] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   02:37:55
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442490/

[5442491] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:22
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442491/

[5442492] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:29:50
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442492/

[5442493] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:12:53
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442493/

[5442494] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:47:50
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442494/

[5442496] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:34:20
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442496/

[5442497] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:45:34
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442497/

[5442498] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:30:32
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442498/

[5442499] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:27:23
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442499/

[5442500] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:23:21
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442500/

[5442501] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:27:38
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442501/

[5442503] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:27:51
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442503/

[5442504] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
time:   01:03:45
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442504/

[5442506] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:59:34
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442506/

[5442507] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:16:43
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442507/

[5442509] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:27:22
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442509/

[5442510] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   01:01:40
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442510/

[5442512] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:12:06
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442512/

[5442514] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:26:39
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442514/

[5442515] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:28
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442515/

[5442516] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:23:37
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442516/

[5442517] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:13:08
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442517/

[5442519] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:22:07
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442519/

[5442522] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   00:54:56
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442522/

[5442523] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:18
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442523/

[5442524] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:00:02
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442524/

[5442525] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:31:50
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442525/

[5442526] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:58:05
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442526/

[5442528] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:39:57
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442528/

[5442529] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:29:01
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442529/

[5442530] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:14:36
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442530/

[5442531] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:33
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442531/

[5442532] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:45:32
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442532/

[5442533] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:13:50
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442533/

[5442535] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:21:20
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442535/

[5442538] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:13:46
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442538/

[5442539] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:13
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442539/

[5442540] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:41:13
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442540/

[5442541] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:13:50
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442541/

[5442542] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:37:29
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442542/

[5442543] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:17:50
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442543/

[5442544] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   00:53:15
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442544/

[5442545] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:32:03
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442545/

[5442546] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:20:46
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442546/

[5442547] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:14:01
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442547/

[5442548] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:32:07
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442548/

[5442549] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:12:17
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442549/

[5442551] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:20:25
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442551/

[5442552] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
time:   00:13:33
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442552/

[5442554] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   03:45:22
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442554/

[5442555] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:48
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442555/

[5442556] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:04:16
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442556/

[5442557] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:15:29
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442557/

[5442558] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:29:23
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442558/

[5442559] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:20:00
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442559/

[5442560] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:29:10
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442560/

[5442561] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   02:22:03
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442561/

[5442562] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:15:38
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442562/

[5442563] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:31
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442563/

[5442564] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:44:25
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442564/

[5442565] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:12:21
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442565/

[5442567] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:03
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442567/

[5442568] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
time:   00:55:42
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442568/

[5442570] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:04:14
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442570/

[5442571] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:16:23
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442571/

[5442572] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   02:28:12
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442572/

[5442573] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:13:13
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442573/

[5442574] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:41:26
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442574/

[5442575] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:06
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442575/

[5442576] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:16:33
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442576/

[5442577] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:55:29
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442577/

[5442578] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:20:06
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442578/

[5442579] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:14:56
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442579/

[5442580] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:19:05
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442580/

[5442581] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:12:58
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442581/

[5442583] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:11:48
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442583/

[5442584] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
time:   00:21:16
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442584/

[5442586] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   02:32:37
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442586/

[5442587] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:14:08
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442587/

[5442588] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   00:39:38
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442588/

[5442589] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:20:23
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442589/

[5442590] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:56:00
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442590/

[5442591] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:42
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442591/

[5442592] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:22:27
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442592/

[5442593] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   01:49:38
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442593/

[5442594] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:15:16
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442594/

[5442595] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:14:11
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442595/

[5442596] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:21:51
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442596/

[5442599] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:46
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442599/

[5442602] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   00:39:09
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442602/

[5442603] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:13:54
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442603/

[5442604] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:16:58
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442604/

[5442605] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:15:24
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442605/

[5442606] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:26:36
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442606/

[5442607] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:27
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442607/

[5442608] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:36:35
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442608/

[5442609] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:28:30
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442609/

[5442610] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:15:09
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442610/

[5442611] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:16:31
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442611/

[5442612] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   01:12:05
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442612/

[5442613] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:12:02
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442613/

[5442615] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:10:51
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442615/

[5442616] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
time:   00:18:00
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442616/

[5442618] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   02:42:17
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442618/

[5442619] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:15:13
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442619/

[5442620] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:31:18
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442620/

[5442621] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:13:40
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442621/

[5442622] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:58:28
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442622/

[5442623] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:50
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442623/

[5442624] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   00:58:28
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442624/

[5442625] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:30:09
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442625/

[5442626] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:21:29
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442626/

[5442627] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:13:58
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442627/

[5442628] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:26:18
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442628/

[5442629] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:10:49
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442629/

[5442631] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:10:19
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442631/

[5442632] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
time:   00:54:22
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442632/

[5442634] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:40:17
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442634/

[5442635] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:15:08
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442635/

[5442636] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:59:19
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442636/

[5442637] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:16:16
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442637/

[5442638] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:26:31
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442638/

[5442639] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:48
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442639/

[5442640] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:09:37
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442640/

[5442641] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   01:51:22
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442641/

[5442642] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:17:16
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442642/

[5442643] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:13:45
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442643/

[5442644] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:40:32
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442644/

[5442645] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:38:52
info:   http://pulpito.ceph.com/pdonnell-2020-09-17_01:14:49-multimds-wip-pdonnell-testing-20200916.234456-distro-basic-smithi/5442645/