2021-08-21T13:15:38.203 INFO:root:teuthology version: 1.1.0-321319b1 2021-08-21T13:15:38.206 INFO:teuthology.results:subset: '6098/9999' 2021-08-21T13:15:38.206 INFO:teuthology.results:seed: '1441' 2021-08-21T15:07:19.000 INFO:root:teuthology version: 1.1.0-321319b1 2021-08-21T15:07:19.029 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-08-21T16:09:23.254 INFO:teuthology.results:Tests finished! gathering results... 2021-08-21T16:09:25.294 INFO:scrape:Found 24 jobs 2021-08-21T16:09:29.551 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350323/teuthology.log 2021-08-21T16:09:29.925 INFO:scrape:Found 6 distinct failure reasons 2021-08-21T16:09:29.926 INFO:scrape:Failure: Command failed (workunit test kernel_untar_build.sh) on gibba030 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=198d0981d770ff64db8c30ce63310e99e361dcf5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/kernel_untar_build.sh' 2021-08-21T16:09:29.927 INFO:scrape:1 jobs: ['6350326'] 2021-08-21T16:09:29.927 INFO:scrape:suites: ['begin', 'clusters/3-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/yes', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'multimds/basic/{0-supported-random-distro$/{ubuntu_18.04}', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'q_check_counter/check_counter', 'session_timeout', 'tasks/cfuse_workunit_kernel_untar_build}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-08-21T16:09:29.928 INFO:scrape: 2021-08-21T16:09:29.928 INFO:scrape:Failure: {'Failure object was': {'gibba011.front.sepia.ceph.com': {'msg': 'Failed to update apt cache: ', 'invocation': {'module_args': {'dpkg_options': 'force-confdef,force-confold', 'autoremove': False, 'force': False, 'force_apt_get': False, 'policy_rc_d': 'None', 'package': 'None', 'autoclean': False, 'install_recommends': 'None', 'purge': False, 'allow_unauthenticated': False, 'state': 'present', 'upgrade': 'None', 'update_cache': True, 'default_release': 'None', 'only_upgrade': False, 'deb': 'None', 'cache_valid_time': 0}}, '_ansible_no_log': False, 'attempts': 24, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/lib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', '_ansible_no_log')"} 2021-08-21T16:09:29.929 INFO:scrape:1 jobs: ['6350334'] 2021-08-21T16:09:29.929 INFO:scrape:suites: ['begin', 'clusters/3-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/yes', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'multimds/basic/{0-supported-random-distro$/{centos_8}', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'q_check_counter/check_counter', 'session_timeout', 'tasks/cfuse_workunit_suites_dbench}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-08-21T16:09:29.930 INFO:scrape: 2021-08-21T16:09:29.930 INFO:scrape:Failure: Command failed (workunit test suites/fsx.sh) on gibba040 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=198d0981d770ff64db8c30ce63310e99e361dcf5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2021-08-21T16:09:29.931 INFO:scrape:1 jobs: ['6350318'] 2021-08-21T16:09:29.931 INFO:scrape:suites: ['begin', 'clusters/3-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/yes', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'multimds/basic/{0-supported-random-distro$/{ubuntu_18.04}', 'objectstore-ec/filestore-xfs', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'q_check_counter/check_counter', 'session_timeout', 'tasks/cfuse_workunit_suites_fsx}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-08-21T16:09:29.932 INFO:scrape: 2021-08-21T16:09:29.932 INFO:scrape:Failure: The following counters failed to be set on mds daemons: {'mds.exported', 'mds.imported'} 2021-08-21T16:09:29.932 INFO:scrape:1 jobs: ['6350322'] 2021-08-21T16:09:29.933 INFO:scrape:suites: ['begin', 'clusters/3-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/yes', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'multimds/basic/{0-supported-random-distro$/{ubuntu_18.04}', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/stock/{k-stock', 'q_check_counter/check_counter', 'rhel_8}', 'session_timeout', 'tasks/cephfs_test_exports}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-08-21T16:09:29.933 INFO:scrape: 2021-08-21T16:09:29.934 INFO:scrape:Dead: None 2021-08-21T16:09:29.934 INFO:scrape:1 jobs: ['6350323'] 2021-08-21T16:09:29.935 INFO:scrape: 2021-08-21T16:09:29.935 INFO:scrape:Failure: reached maximum tries (50) after waiting for 300 seconds 2021-08-21T16:09:29.935 INFO:scrape:1 jobs: ['6350315'] 2021-08-21T16:09:29.936 INFO:scrape:suites: ['centos_latest', 'clusters/9-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount/fuse', 'multimds/verify/{begin', 'objectstore-ec/bluestore-bitmap', 'osd}', 'overrides/{fuse-default-perm-no', 'session_timeout', 'tasks/cfuse_workunit_suites_dbench', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}'] 2021-08-21T16:09:29.936 INFO:scrape: 2021-08-21T16:09:29.938 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba ================================================================= info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/ failed: 4 dead: 2 running: 0 waiting: 0 queued: 0 passed: 18 Fail ================================================================= [6350315] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind} ----------------------------------------------------------------- time: 00:54:24 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350315/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350315/ reached maximum tries (50) after waiting for 300 seconds [6350318] multimds/basic/{0-supported-random-distro$/{ubuntu_18.04} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx} ----------------------------------------------------------------- time: 00:08:10 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350318/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350318/ sentry: https://sentry.ceph.com/organizations/ceph/?query=67f226d7a6cf4816915b0bd240a05aae Command failed (workunit test suites/fsx.sh) on gibba040 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=198d0981d770ff64db8c30ce63310e99e361dcf5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' [6350322] multimds/basic/{0-supported-random-distro$/{ubuntu_18.04} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports} ----------------------------------------------------------------- time: 00:34:48 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350322/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350322/ The following counters failed to be set on mds daemons: {'mds.exported', 'mds.imported'} [6350326] multimds/basic/{0-supported-random-distro$/{ubuntu_18.04} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build} ----------------------------------------------------------------- time: 00:15:57 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350326/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350326/ sentry: https://sentry.ceph.com/organizations/ceph/?query=50efedeffe4b4b16b53e0785e560a2b6 Command failed (workunit test kernel_untar_build.sh) on gibba030 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=198d0981d770ff64db8c30ce63310e99e361dcf5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/kernel_untar_build.sh' Dead ================================================================= [6350323] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350323/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350323/ Error reimaging machines: reached maximum tries (100) after waiting for 600 seconds [6350334] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench} ----------------------------------------------------------------- time: 00:03:22 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350334/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350334/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ace2db2f19024865969eaa2d0d5a6044 {'Failure object was': {'gibba011.front.sepia.ceph.com': {'msg': 'Failed to update apt cache: ', 'invocation': {'module_args': {'dpkg_options': 'force- confdef,force-confold', 'autoremove': False, 'force': False, 'force_apt_get': False, 'policy_rc_d': 'None', 'package': 'None', 'autoclean': False, 'install_recommends': 'None', 'purge': False, 'allow_unauthenticated': False, 'state': 'present', 'upgrade': 'None', 'update_cache': True, 'default_release': 'None', 'only_upgrade': False, 'deb': 'None', 'cache_valid_time': 0}}, '_ansible_no_log': False, 'attempts': 24, 'changed': False}}, 'Traceback (most recent call last)': 'File "/home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/callback_plugins/failure_log.py", line 44, in log_failure log.error(yaml.safe_dump(failure)) File "/home/teuthworker/src/git.ceph.com _git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/lib/pyt hon3.6/site-packages/yaml/__init__.py", line 306, in safe_dump return dump_all([data], stream, Dumper=SafeDumper, **kwds) File "/home/teuthworker /src/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/v irtualenv/lib/python3.6/site-packages/yaml/__init__.py", line 278, in dump_all dumper.represent(data) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 27, in represent node = self.represent_data(data) File "/home/teuthworker/src/git.ceph.com_git_teut hology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/lib/python3.6/si te-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4f f9b63c7655015a3156de2c3b279/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 118, in represent_mapping node_value = self.represent_data(item_value) File "/home/teuthworker/src/git.ceph.com_gi t_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/lib/python 3.6/site-packages/yaml/representer.py", line 48, in represent_data node = self.yaml_representers[data_types[0]](self, data) File "/home/teuthworker/s rc/git.ceph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/vir tualenv/lib/python3.6/site-packages/yaml/representer.py", line 207, in represent_dict return self.represent_mapping(\'tag:yaml.org,2002:map\', data) File "/home/teuthworker/src/git.ceph.com_git_teuthology_321319b12ea4f f9b63c7655015a3156de2c3b279/virtualenv/lib/python3.6/site- packages/yaml/representer.py", line 117, in represent_mapping node_key = self.represent_data(item_key) File "/home/teuthworker/src/git.ceph.com_git_ teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/lib/python3. 6/site-packages/yaml/representer.py", line 58, in represent_data node = self.yaml_representers[None](self, data) File "/home/teuthworker/src/git.ce ph.com_git_teuthology_321319b12ea4ff9b63c7655015a3156de2c3b279/virtualenv/l ib/python3.6/site-packages/yaml/representer.py", line 231, in represent_undefined raise RepresenterError("cannot represent an object", data)', 'yaml.representer.RepresenterError': "('cannot represent an object', '_ansible_no_log')"} Pass ================================================================= [6350314] multimds/basic/{0-supported-random-distro$/{ubuntu_18.04} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb} time: 00:31:30 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350314/ [6350316] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress} time: 00:13:42 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350316/ [6350317] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress} time: 00:11:48 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350317/ [6350319] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep} time: 00:13:59 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350319/ [6350320] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd} time: 00:09:58 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350320/ [6350321] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd} time: 00:11:46 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350321/ [6350324] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots} time: 00:22:52 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350324/ [6350325] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress} time: 00:10:46 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350325/ [6350327] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep} time: 00:12:05 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350327/ [6350328] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc} time: 00:54:15 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350328/ [6350329] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd} time: 00:11:25 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350329/ [6350330] multimds/basic/{0-supported-random-distro$/{ubuntu_18.04} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats} time: 00:11:10 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350330/ [6350331] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind} time: 00:49:14 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350331/ [6350332] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench} time: 00:38:15 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350332/ [6350333] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress} time: 00:13:54 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350333/ [6350335] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep} time: 01:11:48 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350335/ [6350336] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb} time: 01:04:18 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350336/ [6350337] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd} time: 00:11:38 info: http://pulpito.ceph.com/teuthology-2021-08-21_13:15:03-multimds-octopus-distro-basic-gibba/6350337/