2021-03-28T14:40:08.718 INFO:root:teuthology version: 1.1.0-d5a60f86 2021-03-28T14:40:08.719 INFO:teuthology.results:subset: '21/32' 2021-03-28T14:40:08.719 INFO:teuthology.results:seed: '8333' 2021-03-29T04:27:19.546 INFO:root:teuthology version: 1.1.0-d5a60f86 2021-03-29T04:27:19.581 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-03-29T16:27:43.766 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2021-03-29T16:28:52.227 INFO:scrape:Found 260 jobs 2021-03-29T16:29:28.276 INFO:scrape:Found 19 distinct failure reasons 2021-03-29T16:29:28.277 INFO:scrape:Failure: Command failed (workunit test suites/pjd.sh) on smithi110 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=edca387074b365e1c9c97a46f87d28c479c3ff35 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' 2021-03-29T16:29:28.277 INFO:scrape:1 jobs: ['6005456'] 2021-03-29T16:29:28.277 INFO:scrape:suites: ['2-workunit/suites/pjd}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/thrash/workloads/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/none', 'objectstore-ec/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-29T16:29:28.278 INFO:scrape: 2021-03-29T16:29:28.278 INFO:scrape:Failure: Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi003 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=edca387074b365e1c9c97a46f87d28c479c3ff35 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' 2021-03-29T16:29:28.278 INFO:scrape:2 jobs: ['6005476', '6005586'] 2021-03-29T16:29:28.278 INFO:scrape:suites intersection: ['2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'fs/thrash/workloads/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'osd}', 'overrides/{frag_enable', 'session_timeout', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-29T16:29:28.278 INFO:scrape:suites union: ['2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/thrash/workloads/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'ranks/1', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-29T16:29:28.278 INFO:scrape: 2021-03-29T16:29:28.278 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/suites/dbench.sh 2021-03-29T16:29:28.279 INFO:scrape:1 jobs: ['6005513'] 2021-03-29T16:29:28.279 INFO:scrape:suites: ['clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu/{latest', 'distro~HEAD/{ubuntu_latest}', 'fs/verify/{begin', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{frag_enable', 'overrides}}', 'ranks/3', 'session_timeout', 'tasks/dbench', 'validater/valgrind}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-29T16:29:28.279 INFO:scrape: 2021-03-29T16:29:28.279 INFO:scrape:Dead: b'2021-03-29T03:19:02.786 DEBUG:teuthology.exit:Finished running handlers' 2021-03-29T16:29:28.279 INFO:scrape:3 jobs: ['6005502', '6005683', '6005481'] 2021-03-29T16:29:28.279 INFO:scrape:suites intersection: ['clusters/1a5s-mds-1c-client', 'conf/{client', 'mds', 'mon', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{frag_enable', 'session_timeout', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-29T16:29:28.279 INFO:scrape:suites union: ['2-workunit/fs/snaps}}', '2-workunit/suites/fsstress}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu/{latest', 'distro~HEAD/{centos_8}', 'fs/thrash/workloads/{begin', 'fs/verify/{begin', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'mount/kclient/{mount', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'overrides}}', 'ranks/1', 'ranks/5', 'rhel_8}', 'session_timeout', 'tasks/dbench', 'tasks/{1-thrash/osd', 'thrashosds-health', 'validater/valgrind}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-29T16:29:28.280 INFO:scrape: 2021-03-29T16:29:28.280 INFO:scrape:Failure: 'NoneType' object has no attribute 'get' 2021-03-29T16:29:28.280 INFO:scrape:7 jobs: ['6005607', '6005611', '6005620', '6005652', '6005542', '6005689', '6005578'] 2021-03-29T16:29:28.280 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'overrides/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-29T16:29:28.280 INFO:scrape:suites union: ['1-client', '2-upgrade', '2-workunit/cfuse_workunit_suites_pjd}}', '3-client-upgrade', '3-compat_client/pacific}}', '4-compat_client', '5-client-sanity}}', 'clusters/1-mds-2-client-micro', 'clusters/1a3s-mds-2c-client', 'clusters/1a3s-mds-4c-client', 'clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'fs/functional/{begin', 'fs/thrash/multifs/{begin', 'fs/upgrade/featureful_client/old_client/{bluestore-bitmap', 'fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/none', 'multifs', 'multimds/no', 'multimds/yes', 'no_client_pidfile', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore/bluestore-bitmap', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'pg-warn', 'ranks/1', 'rhel_8}', 'scrub/yes', 'session_timeout', 'tasks/acls}', 'tasks/{0-check-counter', 'tasks/{0-nautilus', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/blogbench}}'] 2021-03-29T16:29:28.280 INFO:scrape: 2021-03-29T16:29:28.280 INFO:scrape:Failure: "2021-03-28T15:29:58.998804+0000 mds.d (mds.0) 44 : cluster [WRN] Scrub error on inode 0x1000000689e (/client.0/tmp/payload.1/multiple_rsync_payload.144290/.build-id) see mds.d log and `damage ls` output for details" in cluster log 2021-03-29T16:29:28.280 INFO:scrape:12 jobs: ['6005475', '6005526', '6005442', '6005691', '6005591', '6005655', '6005563', '6005602', '6005521', '6005686', '6005650', '6005617'] 2021-03-29T16:29:28.281 INFO:scrape:suites intersection: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'fs/workload/{begin', 'mds', 'mon', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{frag_enable', 'scrub/yes', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-29T16:29:28.281 INFO:scrape:suites union: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/fuse', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'ranks/3', 'ranks/5', 'rhel_8}', 'scrub/yes', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/misc}}', 'workunit/suites/blogbench}}', 'workunit/suites/ffsb}}', 'workunit/suites/fsstress}}', 'workunit/suites/pjd}}'] 2021-03-29T16:29:28.281 INFO:scrape: 2021-03-29T16:29:28.281 INFO:scrape:Failure: "2021-03-28T15:37:09.127845+0000 mds.a (mds.0) 24 : cluster [WRN] Scrub error on inode 0x10000000435 (/client.0/tmp/tmp/fstest_7b3e284c14962b9eeb49b0eabe4d45ad/fstest_d1dbb62b36d590be520824bd9780d843) see mds.a log and `damage ls` output for details" in cluster log 2021-03-29T16:29:28.281 INFO:scrape:1 jobs: ['6005508'] 2021-03-29T16:29:28.281 INFO:scrape:suites: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-bitmap', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'ranks/3', 'scrub/yes', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/pjd}}'] 2021-03-29T16:29:28.282 INFO:scrape: 2021-03-29T16:29:28.282 INFO:scrape:Crash: Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi008 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=edca387074b365e1c9c97a46f87d28c479c3ff35 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' 2021-03-29T16:29:28.282 INFO:scrape:ceph version 16.1.0-1306-gedca3870 (edca387074b365e1c9c97a46f87d28c479c3ff35) pacific (rc) 1: /lib64/libpthread.so.0(+0x12dc0) [0x7f5e24fd2dc0] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x7f5e26226399] 5: /usr/lib64/ceph/libceph-common.so.2(+0x276562) [0x7f5e26226562] 6: ceph-mds(+0x26aef3) [0x56054a7f0ef3] 7: (MDCache::dispatch_fragment_dir(boost::intrusive_ptr&)+0x4d3) [0x56054a880023] 8: (MDCache::fragment_frozen(boost::intrusive_ptr&, int)+0x2cc) [0x56054a880d4c] 9: (MDCache::fragment_mark_and_complete(boost::intrusive_ptr&)+0xa2c) [0x56054a89094c] 10: (MDCache::merge_dir(CInode*, frag_t)+0x590) [0x56054a891740] 11: ceph-mds(+0x3e5562) [0x56054a96b562] 12: (Context::complete(int)+0xd) [0x56054a73aa0d] 13: (SafeTimer::timer_thread()+0x1b7) [0x7f5e26314b77] 14: (SafeTimerThread::entry()+0x11) [0x7f5e26316151] 15: /lib64/libpthread.so.0(+0x82de) [0x7f5e24fc82de] 16: clone() 2021-03-29T16:29:28.282 INFO:scrape:1 jobs: ['6005646'] 2021-03-29T16:29:28.282 INFO:scrape:suites: ['2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/thrash/workloads/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-29T16:29:28.282 INFO:scrape: 2021-03-29T16:29:28.282 INFO:scrape:Failure: Test failure: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2021-03-29T16:29:28.282 INFO:scrape:1 jobs: ['6005582'] 2021-03-29T16:29:28.282 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/snap-schedule}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-29T16:29:28.283 INFO:scrape: 2021-03-29T16:29:28.283 INFO:scrape:Failure: Test failure: test_object_deletion (tasks.cephfs.test_damage.TestDamage) 2021-03-29T16:29:28.283 INFO:scrape:1 jobs: ['6005687'] 2021-03-29T16:29:28.283 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/damage}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-29T16:29:28.284 INFO:scrape: 2021-03-29T16:29:28.284 INFO:scrape:Failure: Command failed on smithi136 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2021-03-29T16:29:28.284 INFO:scrape:1 jobs: ['6005505'] 2021-03-29T16:29:28.284 INFO:scrape:suites: ['1-client', '2-upgrade', '3-verify}', 'clusters/1-mds-2-client-micro', 'conf/{client', 'fs/upgrade/volumes/import-legacy/{bluestore-bitmap', 'mds', 'mon', 'osd}', 'overrides/{frag_enable', 'pg-warn', 'tasks/{0-nautilus', 'ubuntu_18.04}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-29T16:29:28.284 INFO:scrape: 2021-03-29T16:29:28.284 INFO:scrape:Failure: "2021-03-28T18:22:08.312684+0000 mds.f (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000001fb (/client.0/tmp/fsstress/ltp-full-20091231/testcases/ballista) see mds.f log and `damage ls` output for details" in cluster log 2021-03-29T16:29:28.284 INFO:scrape:1 jobs: ['6005575'] 2021-03-29T16:29:28.285 INFO:scrape:suites: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{rhel_8}', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'ranks/5', 'scrub/yes', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/fsstress}}'] 2021-03-29T16:29:28.285 INFO:scrape: 2021-03-29T16:29:28.285 INFO:scrape:Failure: Command failed (workunit test fs/quota/quota.sh) on smithi193 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.3/client.3/tmp && cd -- /home/ubuntu/cephtest/mnt.3/client.3/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=edca387074b365e1c9c97a46f87d28c479c3ff35 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="3" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.3 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.3/qa/workunits/fs/quota/quota.sh' 2021-03-29T16:29:28.285 INFO:scrape:1 jobs: ['6005616'] 2021-03-29T16:29:28.285 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'tasks/workunit/quota}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-29T16:29:28.285 INFO:scrape: 2021-03-29T16:29:28.285 INFO:scrape:Failure: "2021-03-28T15:24:16.221679+0000 mds.c (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000003b1 (/client.0/tmp/fsstress/ltp-full-20091231/testcases/commands) see mds.c log and `damage ls` output for details" in cluster log 2021-03-29T16:29:28.285 INFO:scrape:1 jobs: ['6005489'] 2021-03-29T16:29:28.285 INFO:scrape:suites: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{rhel_8}', 'fs/workload/{begin', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-comp', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{frag_enable', 'ranks/3', 'scrub/yes', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/fsstress}}'] 2021-03-29T16:29:28.286 INFO:scrape: 2021-03-29T16:29:28.286 INFO:scrape:Failure: Test failure: test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth) 2021-03-29T16:29:28.286 INFO:scrape:2 jobs: ['6005466', '6005682'] 2021-03-29T16:29:28.286 INFO:scrape:suites intersection: ['clusters/1a3s-mds-2c-client', 'conf/{client', 'fs/multifs/{begin', 'k-testing}', 'mds', 'mon', 'mon-debug', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/multifs-auth}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-29T16:29:28.286 INFO:scrape:suites union: ['clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/multifs/{begin', 'k-testing}', 'mds', 'mon', 'mon-debug', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/multifs-auth}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-29T16:29:28.286 INFO:scrape: 2021-03-29T16:29:28.286 INFO:scrape:Failure: "2021-03-28T16:09:31.513073+0000 mds.b (mds.3) 18 : cluster [WRN] Scrub error on inode 0x10000000266 (/client.0/tmp/testdir/dir1/dir2/dir3/dir4) see mds.b log and `damage ls` output for details" in cluster log 2021-03-29T16:29:28.286 INFO:scrape:1 jobs: ['6005558'] 2021-03-29T16:29:28.287 INFO:scrape:suites: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{rhel_8}', 'fs/workload/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'ranks/5', 'rhel_8}', 'scrub/yes', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/misc}}'] 2021-03-29T16:29:28.287 INFO:scrape: 2021-03-29T16:29:28.287 INFO:scrape:Failure: Test failure: test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes), test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes) 2021-03-29T16:29:28.287 INFO:scrape:1 jobs: ['6005527'] 2021-03-29T16:29:28.287 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/volumes/{overrides', 'test/basic}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-29T16:29:28.288 INFO:scrape: 2021-03-29T16:29:28.288 INFO:scrape:Failure: The following counters failed to be set on mds daemons: {'mds.exported', 'mds.imported'} 2021-03-29T16:29:28.288 INFO:scrape:1 jobs: ['6005503'] 2021-03-29T16:29:28.288 INFO:scrape:suites: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{rhel_8}', 'fs/workload/{begin', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-comp', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{frag_enable', 'ranks/5', 'scrub/yes', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/iogen}}'] 2021-03-29T16:29:28.288 INFO:scrape: 2021-03-29T16:29:28.288 INFO:scrape:Failure: Test failure: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2021-03-29T16:29:28.288 INFO:scrape:1 jobs: ['6005525'] 2021-03-29T16:29:28.288 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/mds-full}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-29T16:29:28.289 INFO:scrape: 2021-03-29T16:29:28.304 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/ failed: 37 dead: 2 running: 1 waiting: 0 queued: 0 passed: 220 Fail ================================================================= [6005442] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:20:16 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005442/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005442/ "2021-03-28T15:05:53.577993+0000 mds.f (mds.0) 19 : cluster [WRN] Scrub error on inode 0x100000030aa (/client.0/tmp/tmp.BOcv9mjtNl/p1) see mds.f log and `damage ls` output for details" in cluster log [6005456] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:16:06 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005456/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005456/ sentry: https://sentry.ceph.com/organizations/ceph/?query=404dd1b6eab4476bad94335f2b12c7c2 Command failed (workunit test suites/pjd.sh) on smithi110 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=edca387074b365e1c9c97a46f87d28c479c3ff35 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6005466] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} ----------------------------------------------------------------- time: 00:10:55 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005466/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005466/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bbff03d2258b448389ec49ef5bc82fe5 Test failure: test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth) [6005475] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/fs/misc}} ----------------------------------------------------------------- time: 00:36:40 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005475/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005475/ "2021-03-28T15:29:58.998804+0000 mds.d (mds.0) 44 : cluster [WRN] Scrub error on inode 0x1000000689e (/client.0/tmp/payload.1/multiple_rsync_payload.144290/.build-id) see mds.d log and `damage ls` output for details" in cluster log [6005476] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 01:38:33 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005476/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005476/ sentry: https://sentry.ceph.com/organizations/ceph/?query=27afa9a1b9ea4c8b9f56de9d80c4721b Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi003 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=edca387074b365e1c9c97a46f87d28c479c3ff35 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' [6005489] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:14:57 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005489/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005489/ "2021-03-28T15:24:16.221679+0000 mds.c (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000003b1 (/client.0/tmp/fsstress/ltp- full-20091231/testcases/commands) see mds.c log and `damage ls` output for details" in cluster log [6005503] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/iogen}} ----------------------------------------------------------------- time: 00:54:23 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005503/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005503/ The following counters failed to be set on mds daemons: {'mds.exported', 'mds.imported'} [6005505] fs/upgrade/volumes/import-legacy/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} ----------------------------------------------------------------- time: 01:07:54 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005505/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005505/ Command failed on smithi136 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' [6005508] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:15:30 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005508/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005508/ "2021-03-28T15:37:09.127845+0000 mds.a (mds.0) 24 : cluster [WRN] Scrub error on inode 0x10000000435 (/client.0/tmp/tmp/fstest_7b3e284c14962b9eeb49 b0eabe4d45ad/fstest_d1dbb62b36d590be520824bd9780d843) see mds.a log and `damage ls` output for details" in cluster log [6005513] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} distro~HEAD/{ubuntu_latest} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/dbench validater/valgrind} ----------------------------------------------------------------- time: 03:45:52 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005513/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005513/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e6d2fee37d934284939a9244ad32d440 Command failed (workunit test suites/dbench.sh) on smithi037 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=edca387074b365e1c9c97a46f87d28c479c3ff35 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' [6005521] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} ----------------------------------------------------------------- time: 00:29:15 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005521/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005521/ "2021-03-28T15:46:01.586813+0000 mds.a (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000001fe (/client.0/tmp/blogbench-1.0/src) see mds.a log and `damage ls` output for details" in cluster log [6005525] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-full} ----------------------------------------------------------------- time: 00:16:50 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005525/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005525/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a2d862ed205c45d2b61c14a0a9987fac Test failure: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) [6005526] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:48:05 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005526/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005526/ "2021-03-28T15:49:39.094569+0000 mds.c (mds.0) 14 : cluster [WRN] Scrub error on inode 0x10000000347 (/client.0/tmp/tmp) see mds.c log and `damage ls` output for details" in cluster log [6005527] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} ----------------------------------------------------------------- time: 00:48:34 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005527/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005527/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2f1249172f0b48fa8eb366eb48326b6b Test failure: test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes), test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes) [6005542] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} ----------------------------------------------------------------- time: 00:13:05 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005542/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005542/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3ec5159fb794453292b92eb5c47863a0 'NoneType' object has no attribute 'mon_manager' [6005558] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/fs/misc}} ----------------------------------------------------------------- time: 00:36:41 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005558/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005558/ "2021-03-28T16:09:31.513073+0000 mds.b (mds.3) 18 : cluster [WRN] Scrub error on inode 0x10000000266 (/client.0/tmp/testdir/dir1/dir2/dir3/dir4) see mds.b log and `damage ls` output for details" in cluster log [6005563] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} ----------------------------------------------------------------- time: 00:24:11 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005563/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005563/ "2021-03-28T18:06:52.450098+0000 mds.d (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000001fa (/client.0/tmp/blogbench-1.0/man) see mds.d log and `damage ls` output for details" in cluster log [6005575] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:16:16 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005575/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005575/ "2021-03-28T18:22:08.312684+0000 mds.f (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000001fb (/client.0/tmp/fsstress/ltp- full-20091231/testcases/ballista) see mds.f log and `damage ls` output for details" in cluster log [6005578] fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}} ----------------------------------------------------------------- time: 00:18:57 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005578/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005578/ sentry: https://sentry.ceph.com/organizations/ceph/?query=310f37405ea54b5a90887df7da8b0a4b 'NoneType' object has no attribute 'mon_manager' [6005582] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snap-schedule} ----------------------------------------------------------------- time: 00:12:10 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005582/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005582/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2780d9f1418c43119a4abd37b2cd64e6 Test failure: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) [6005586] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 01:40:48 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005586/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005586/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f7f32bb5691d4ef786daff308fe8539f Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi013 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=edca387074b365e1c9c97a46f87d28c479c3ff35 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' [6005591] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:13:28 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005591/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005591/ "2021-03-28T18:29:37.284341+0000 mds.c (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000002cd (/client.0/tmp/tmp) see mds.c log and `damage ls` output for details" in cluster log [6005602] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/fs/misc}} ----------------------------------------------------------------- time: 00:59:27 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005602/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005602/ "2021-03-28T18:44:54.352669+0000 mds.f (mds.0) 19 : cluster [WRN] Scrub error on inode 0x10000000264 (~mds0/stray8/10000000264) see mds.f log and `damage ls` output for details" in cluster log [6005607] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} ----------------------------------------------------------------- time: 00:14:56 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005607/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005607/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1bbe2d7e4488418e90da595a9ed3a525 'NoneType' object has no attribute 'get' [6005611] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005611/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005611/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3d010907819248e68d0deeeb32709f1b 'NoneType' object has no attribute 'get' [6005616] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/quota} ----------------------------------------------------------------- time: 00:12:15 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005616/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005616/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1478a6a7445246ffb09c1ec03f6c45d7 Command failed (workunit test fs/quota/quota.sh) on smithi193 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.3/client.3/tmp && cd -- /home/ubuntu/cephtest/mnt.3/client.3/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=edca387074b365e1c9c97a46f87d28c479c3ff35 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="3" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.3 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.3/qa/workunits/fs/quota/quota.sh' [6005617] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:17:22 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005617/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005617/ "2021-03-28T18:50:34.554217+0000 mds.f (mds.0) 19 : cluster [WRN] Scrub error on inode 0x100000030aa (/client.0/tmp/tmp.gTr0ieM8fh/p2) see mds.f log and `damage ls` output for details" in cluster log [6005620] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/acls} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005620/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005620/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3f7d7a791bbe4120a83d86712019265e 'NoneType' object has no attribute 'get' [6005646] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 02:19:53 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005646/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005646/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2686fb0e567246f997ae20bad3d88316 Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi008 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=edca387074b365e1c9c97a46f87d28c479c3ff35 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' [6005650] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} ----------------------------------------------------------------- time: 00:27:04 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005650/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005650/ "2021-03-28T19:17:15.253280+0000 mds.d (mds.0) 19 : cluster [WRN] Scrub error on inode 0x1000000039f (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-0) see mds.d log and `damage ls` output for details" in cluster log [6005652] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} ----------------------------------------------------------------- time: 00:14:18 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005652/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005652/ sentry: https://sentry.ceph.com/organizations/ceph/?query=33828a64413345b0be3f4e750fb80c45 'NoneType' object has no attribute 'mon_manager' [6005655] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:20:54 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005655/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005655/ "2021-03-28T19:24:17.958712+0000 mds.c (mds.0) 54 : cluster [WRN] Scrub error on inode 0x10000000348 (/client.0/tmp/tmp/data) see mds.c log and `damage ls` output for details" in cluster log [6005682] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} ----------------------------------------------------------------- time: 00:11:05 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005682/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005682/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1b82c2d3f85a4b69958eca3b4e6b4f54 Test failure: test_mount_all_caps_absent (tasks.cephfs.test_multifs_auth.TestClientsWithoutAuth) [6005686] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/fs/misc}} ----------------------------------------------------------------- time: 00:43:49 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005686/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005686/ "2021-03-29T04:42:57.527883+0000 mds.b (mds.3) 4 : cluster [WRN] Scrub error on inode 0x10000000264 (/client.0/tmp/testdir/dir1/dir2) see mds.b log and `damage ls` output for details" in cluster log [6005687] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/damage} ----------------------------------------------------------------- time: 00:20:55 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005687/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005687/ sentry: https://sentry.ceph.com/organizations/ceph/?query=330635aa9df14e148133755b21a32f09 Test failure: test_object_deletion (tasks.cephfs.test_damage.TestDamage) [6005689] fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}} ----------------------------------------------------------------- time: 00:19:52 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005689/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005689/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4c4b5b0a1d8544faa296306f88525260 'NoneType' object has no attribute 'mon_manager' [6005691] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} ----------------------------------------------------------------- time: 00:28:16 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005691/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005691/ "2021-03-29T04:41:08.232316+0000 mds.f (mds.0) 14 : cluster [WRN] Scrub error on inode 0x10000000df1 (/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-13) see mds.f log and `damage ls` output for details" in cluster log Dead ================================================================= [6005481] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005481/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005481/ hit max job timeout [6005502] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005502/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005502/ hit max job timeout Running ================================================================= [6005683] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} distro~HEAD/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/dbench validater/valgrind} info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005683/ Pass ================================================================= [6005438] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:54:37 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005438/ [6005439] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/fs/norstats}} time: 00:13:04 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005439/ [6005440] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:20:20 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005440/ [6005441] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:15:36 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005441/ [6005443] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:22:19 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005443/ [6005444] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/backtrace} time: 00:10:53 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005444/ [6005445] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/suites/fsx}} time: 00:21:40 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005445/ [6005446] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:24:08 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005446/ [6005447] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/fsync-tester}} time: 00:26:38 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005447/ [6005448] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:19:32 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005448/ [6005449] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cap-flush} time: 00:09:13 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005449/ [6005450] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:09:47 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005450/ [6005451] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc}} time: 00:23:08 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005451/ [6005452] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/iogen}} time: 00:54:26 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005452/ [6005453] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:14:52 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005453/ [6005454] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-limits} time: 00:17:33 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005454/ [6005455] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/suites/iozone}} time: 00:15:27 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005455/ [6005457] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/pjd}} time: 00:12:39 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005457/ [6005458] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:12:52 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005458/ [6005459] fs/bugs/client_trim_caps/{begin clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/trim-i22073} time: 00:08:09 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005459/ [6005460] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/centos_latest} time: 00:44:04 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005460/ [6005461] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-readahead} time: 00:13:22 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005461/ [6005462] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client} time: 00:08:22 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005462/ [6005463] fs/mirror/{begin cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{rhel_8} tasks/mirror} time: 01:03:04 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005463/ [6005464] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 01:24:46 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005464/ [6005465] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:18:35 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005465/ [6005467] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:11:44 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005467/ [6005468] fs/shell/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cephfs-shell} time: 00:32:56 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005468/ [6005469] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} time: 00:13:31 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005469/ [6005470] fs/top/{begin cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/whitelist_health supported-random-distros$/{rhel_8} tasks/fstop} time: 00:10:20 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005470/ [6005471] fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}} time: 00:28:11 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005471/ [6005472] fs/valgrind/{begin centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror}} time: 01:15:02 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005472/ [6005473] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} distro~HEAD/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/fsstress validater/lockdep} time: 00:14:09 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005473/ [6005474] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/direct_io}} time: 00:12:20 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005474/ [6005477] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/kernel_untar_build}} time: 00:43:34 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005477/ [6005478] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery} time: 00:30:40 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005478/ [6005479] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:45:17 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005479/ [6005480] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} time: 00:17:40 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005480/ [6005482] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/suites/dbench}} time: 01:22:38 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005482/ [6005483] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/damage} time: 00:20:28 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005483/ [6005484] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/ffsb}} time: 00:21:53 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005484/ [6005485] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:13:53 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005485/ [6005486] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/fs/norstats}} time: 00:13:28 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005486/ [6005487] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/data-scan} time: 00:26:02 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005487/ [6005488] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:20:22 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005488/ [6005490] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:58:11 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005490/ [6005491] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} time: 00:23:12 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005491/ [6005492] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} time: 00:24:55 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005492/ [6005493] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:13:32 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005493/ [6005494] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/suites/fsx}} time: 00:22:51 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005494/ [6005495] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/exports} time: 00:49:48 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005495/ [6005496] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/fsync-tester}} time: 00:14:31 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005496/ [6005497] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/ffsb}} time: 00:22:50 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005497/ [6005498] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc}} time: 00:20:44 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005498/ [6005499] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:12:44 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005499/ [6005500] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub} time: 00:15:32 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005500/ [6005501] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:11:38 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005501/ [6005504] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:21:03 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005504/ [6005506] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/suites/iozone}} time: 00:13:33 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005506/ [6005507] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/fragment} time: 00:14:56 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005507/ [6005509] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:22:44 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005509/ [6005510] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 01:24:04 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005510/ [6005511] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:09:31 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005511/ [6005512] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:42:39 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005512/ [6005514] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/direct_io}} time: 00:13:09 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005514/ [6005515] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:09:03 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005515/ [6005516] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/journal-repair} time: 00:17:05 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005516/ [6005517] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/fs/misc}} time: 00:55:37 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005517/ [6005518] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:13:12 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005518/ [6005519] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/kernel_untar_build}} time: 00:43:25 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005519/ [6005520] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush} time: 00:13:11 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005520/ [6005522] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:15:45 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005522/ [6005523] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/suites/dbench}} time: 00:56:56 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005523/ [6005524] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/iozone}} time: 00:14:33 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005524/ [6005528] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:36:02 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005528/ [6005529] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/fs/norstats}} time: 00:14:22 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005529/ [6005530] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:15:57 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005530/ [6005531] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds_creation_retry} time: 00:10:54 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005531/ [6005532] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:11:24 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005532/ [6005533] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/fsstress}} time: 00:16:21 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005533/ [6005534] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:51:40 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005534/ [6005535] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/suites/fsx}} time: 00:28:45 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005535/ [6005536] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:24:51 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005536/ [6005537] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:35:57 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005537/ [6005538] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} time: 00:47:04 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005538/ [6005539] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/fsync-tester}} time: 00:12:36 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005539/ [6005540] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/metrics} time: 00:19:36 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005540/ [6005541] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:13:14 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005541/ [6005543] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc}} time: 00:22:06 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005543/ [6005544] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/iogen}} time: 00:25:24 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005544/ [6005545] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/multimds_misc} time: 00:21:31 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005545/ [6005546] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:19:58 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005546/ [6005547] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/suites/iozone}} time: 00:16:23 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005547/ [6005548] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:11:15 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005548/ [6005549] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/pjd}} time: 00:11:22 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005549/ [6005550] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/openfiletable} time: 00:12:42 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005550/ [6005551] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs} time: 00:13:02 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005551/ [6005552] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 00:39:26 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005552/ [6005553] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} time: 00:10:34 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005553/ [6005554] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:40:57 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005554/ [6005555] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} time: 02:13:22 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005555/ [6005556] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} distro~HEAD/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/fsstress validater/lockdep} time: 00:14:12 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005556/ [6005557] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/direct_io}} time: 00:13:16 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005557/ [6005559] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 02:32:54 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005559/ [6005560] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm} time: 00:12:24 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005560/ [6005561] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/kernel_untar_build}} time: 00:35:32 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005561/ [6005562] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:56:06 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005562/ [6005564] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:12:37 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005564/ [6005565] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/quota} time: 00:12:23 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005565/ [6005566] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:10:48 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005566/ [6005567] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:25:16 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005567/ [6005568] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/suites/dbench}} time: 00:43:00 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005568/ [6005569] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:40:42 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005569/ [6005570] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/ffsb}} time: 00:50:18 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005570/ [6005571] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:10:43 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005571/ [6005572] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/scrub} time: 00:23:46 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005572/ [6005573] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/fs/norstats}} time: 00:16:42 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005573/ [6005574] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:11:34 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005574/ [6005576] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/sessionmap} time: 00:14:04 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005576/ [6005577] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:13:20 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005577/ [6005579] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/suites/fsx}} time: 00:17:46 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005579/ [6005580] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/fsync-tester}} time: 00:13:15 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005580/ [6005581] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:22:19 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005581/ [6005583] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc}} time: 00:27:15 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005583/ [6005584] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:57:34 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005584/ [6005585] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} time: 01:13:24 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005585/ [6005587] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/iogen}} time: 00:55:24 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005587/ [6005588] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snapshots} time: 00:21:08 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005588/ [6005589] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:54:32 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005589/ [6005590] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/suites/iozone}} time: 00:12:19 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005590/ [6005592] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:17:28 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005592/ [6005593] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:12:02 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005593/ [6005594] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/strays} time: 00:29:34 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005594/ [6005595] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 01:04:39 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005595/ [6005596] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:18:52 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005596/ [6005597] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:20:06 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005597/ [6005598] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:11:01 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005598/ [6005599] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} distro~HEAD/{rhel_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/dbench validater/valgrind} time: 00:58:27 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005599/ [6005600] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/direct_io}} time: 00:14:40 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005600/ [6005601] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:12:27 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005601/ [6005603] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:19:25 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005603/ [6005604] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/kernel_untar_build}} time: 01:29:31 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005604/ [6005605] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/test_journal_migration} time: 00:16:35 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005605/ [6005606] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:15:00 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005606/ [6005608] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:13:31 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005608/ [6005609] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/suites/dbench}} time: 01:22:35 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005609/ [6005610] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/truncate_delay} time: 00:09:17 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005610/ [6005612] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/ffsb}} time: 00:52:01 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005612/ [6005613] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 04:48:35 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005613/ [6005614] fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}} time: 00:28:13 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005614/ [6005615] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/fs/norstats}} time: 00:13:08 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005615/ [6005618] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 01:04:02 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005618/ [6005619] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/suites/fsx}} time: 00:25:07 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005619/ [6005621] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:23:28 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005621/ [6005622] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/fsync-tester}} time: 00:12:40 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005622/ [6005623] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:11:14 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005623/ [6005624] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc}} time: 00:20:31 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005624/ [6005625] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:16:11 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005625/ [6005626] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/admin} time: 00:37:00 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005626/ [6005627] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:12:37 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005627/ [6005628] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/iogen}} time: 01:11:44 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005628/ [6005629] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:12:33 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005629/ [6005630] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:21:57 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005630/ [6005631] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:15:22 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005631/ [6005632] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/suites/iozone}} time: 00:19:51 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005632/ [6005633] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/alternate-pool} time: 00:12:03 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005633/ [6005634] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:13:09 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005634/ [6005635] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/pjd}} time: 00:14:06 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005635/ [6005636] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs_python} time: 00:10:50 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005636/ [6005637] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 01:13:09 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005637/ [6005638] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:09:52 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005638/ [6005639] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:50:19 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005639/ [6005640] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} time: 00:11:45 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005640/ [6005641] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} distro~HEAD/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/fsstress validater/lockdep} time: 00:16:58 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005641/ [6005642] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/direct_io}} time: 00:14:46 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005642/ [6005643] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:20:34 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005643/ [6005644] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/asok_dump_tree} time: 00:22:31 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005644/ [6005645] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/fs/misc}} time: 00:29:07 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005645/ [6005647] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/kernel_untar_build}} time: 00:42:39 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005647/ [6005648] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:58:56 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005648/ [6005649] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/auto-repair} time: 00:12:05 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005649/ [6005651] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:17:26 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005651/ [6005653] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/suites/dbench}} time: 00:47:46 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005653/ [6005654] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/backtrace} time: 00:13:57 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005654/ [6005656] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:12:28 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005656/ [6005657] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/fs/norstats}} time: 00:15:17 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005657/ [6005658] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:19:01 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005658/ [6005659] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:11:21 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005659/ [6005660] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cap-flush} time: 00:11:31 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005660/ [6005661] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:10:18 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005661/ [6005662] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/fsstress}} time: 00:14:52 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005662/ [6005663] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:13:31 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005663/ [6005664] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/suites/fsx}} time: 01:00:54 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005664/ [6005665] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/fsync-tester}} time: 00:12:48 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005665/ [6005666] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-limits} time: 00:19:56 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005666/ [6005667] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/ffsb}} time: 00:27:46 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005667/ [6005668] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc}} time: 00:23:03 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005668/ [6005669] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 03:14:13 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005669/ [6005670] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/iogen}} time: 00:43:43 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005670/ [6005671] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-readahead} time: 00:10:43 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005671/ [6005672] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:57:46 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005672/ [6005673] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/suites/iozone}} time: 00:13:13 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005673/ [6005674] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:03:26 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005674/ [6005675] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} time: 00:21:22 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005675/ [6005676] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} time: 00:35:03 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005676/ [6005677] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/pjd}} time: 00:12:42 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005677/ [6005678] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery} time: 00:36:27 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005678/ [6005679] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:22:24 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005679/ [6005680] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 01:18:34 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005680/ [6005681] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} time: 00:19:11 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005681/ [6005684] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/direct_io}} time: 00:15:40 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005684/ [6005685] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:10:48 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005685/ [6005688] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:11:22 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005688/ [6005690] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/kernel_untar_build}} time: 01:57:06 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005690/ [6005692] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:13:36 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005692/ [6005693] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:13:49 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005693/ [6005694] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/data-scan} time: 00:28:17 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005694/ [6005695] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:13:10 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005695/ [6005696] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/suites/dbench}} time: 01:11:00 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005696/ [6005697] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/iozone}} time: 00:13:47 info: http://pulpito.ceph.com/teuthology-2021-03-28_04:17:03-fs-pacific-distro-basic-smithi/6005697/