2021-04-24T19:16:09.259 INFO:root:teuthology version: 1.1.0-2713a3cd 2021-04-24T19:16:09.260 INFO:teuthology.results:subset: '2/32' 2021-04-24T19:16:09.260 INFO:teuthology.results:seed: '5755' 2021-04-24T22:20:41.878 INFO:root:teuthology version: 1.1.0-2713a3cd 2021-04-24T22:20:41.902 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-04-25T01:34:41.867 INFO:teuthology.results:Tests finished! gathering results... 2021-04-25T01:35:11.826 INFO:scrape:Found 260 jobs 2021-04-25T01:35:33.160 INFO:scrape:Found 13 distinct failure reasons 2021-04-25T01:35:33.161 INFO:scrape:Failure: "2021-04-24T21:43:08.234625+0000 mds.b (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000002cd (/client.0/tmp/tmp) see mds.b log and `damage ls` output for details" in cluster log 2021-04-25T01:35:33.161 INFO:scrape:14 jobs: ['6067445', '6067423', '6067386', '6067375', '6067401', '6067412', '6067500', '6067454', '6067301', '6067428', '6067514', '6067286', '6067333', '6067338'] 2021-04-25T01:35:33.161 INFO:scrape:suites intersection: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'fs/workload/{begin', 'mds', 'mon', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{frag_enable', 'scrub/yes', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-04-25T01:35:33.161 INFO:scrape:suites union: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/fuse', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'ranks/1', 'ranks/3', 'ranks/5', 'rhel_8}', 'scrub/yes', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/misc}}', 'workunit/suites/blogbench}}', 'workunit/suites/ffsb}}', 'workunit/suites/fsstress}}', 'workunit/suites/pjd}}'] 2021-04-25T01:35:33.162 INFO:scrape: 2021-04-25T01:35:33.162 INFO:scrape:Failure: Test failure: test_delete_object (tasks.cephfs.test_volume_client.TestVolumeClient) 2021-04-25T01:35:33.162 INFO:scrape:2 jobs: ['6067513', '6067329'] 2021-04-25T01:35:33.162 INFO:scrape:suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/volumes/{begin', 'mds', 'mon', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/volume-client}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-04-25T01:35:33.162 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{ubuntu_latest}', 'fs/volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/fuse', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/volume-client}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-04-25T01:35:33.162 INFO:scrape: 2021-04-25T01:35:33.162 INFO:scrape:Failure: Test failure: test_object_deletion (tasks.cephfs.test_damage.TestDamage) 2021-04-25T01:35:33.162 INFO:scrape:1 jobs: ['6067443'] 2021-04-25T01:35:33.163 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/damage}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-04-25T01:35:33.163 INFO:scrape: 2021-04-25T01:35:33.163 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/fs/misc/multiple_rsync.sh 2021-04-25T01:35:33.163 INFO:scrape:2 jobs: ['6067495', '6067368'] 2021-04-25T01:35:33.163 INFO:scrape:suites intersection: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{frag_enable', 'ranks/5', 'scrub/yes', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/misc}}'] 2021-04-25T01:35:33.163 INFO:scrape:suites union: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'ranks/5', 'scrub/yes', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/fs/misc}}'] 2021-04-25T01:35:33.163 INFO:scrape: 2021-04-25T01:35:33.164 INFO:scrape:Crash: Command failed on smithi018 with status 1: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp' 2021-04-25T01:35:33.164 INFO:scrape:ceph version 16.2.1-107-g80c1bb7e (80c1bb7e30aa853a28409be8101c5a37d1e35fd3) pacific (stable) 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0) [0x7f29f29623c0] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1ad) [0x7f29f2e7eea2] 5: /usr/lib/ceph/libceph-common.so.2(+0x26404f) [0x7f29f2e7f04f] 6: (Monitor::~Monitor()+0x144b) [0x557f1132be0b] 7: (Monitor::~Monitor()+0xd) [0x557f1132be8d] 8: main() 9: __libc_start_main() 10: _start() 2021-04-25T01:35:33.164 INFO:scrape:1 jobs: ['6067459'] 2021-04-25T01:35:33.164 INFO:scrape:suites: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'ranks/5', 'scrub/yes', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/blogbench}}'] 2021-04-25T01:35:33.164 INFO:scrape: 2021-04-25T01:35:33.164 INFO:scrape:Crash: Command failed on smithi019 with status 1: "sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/client.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' --id 0 /home/ubuntu/cephtest/mnt.0" 2021-04-25T01:35:33.165 INFO:scrape:ceph version 16.2.1-107-g80c1bb7e (80c1bb7e30aa853a28409be8101c5a37d1e35fd3) pacific (stable) 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7f83cd871980] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x19c) [0x5648ce6009ce] 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x5648ce600b58] 6: ceph-osd(+0xb45dff) [0x5648ce710dff] 7: (OSDService::~OSDService()+0xd9) [0x5648ce7164b9] 8: (OSD::~OSD()+0x217) [0x5648ce6c2387] 9: (OSD::~OSD()+0x9) [0x5648ce6c29e9] 10: main() 11: __libc_start_main() 12: _start() 2021-04-25T01:35:33.165 INFO:scrape:1 jobs: ['6067366'] 2021-04-25T01:35:33.165 INFO:scrape:suites: ['clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu/{latest', 'distro~HEAD/{ubuntu_latest}', 'fs/verify/{begin', 'mds', 'mon', 'mon-debug', 'mount/fuse', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{frag_enable', 'overrides}}', 'ranks/5', 'session_timeout', 'tasks/dbench', 'validater/valgrind}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-04-25T01:35:33.165 INFO:scrape: 2021-04-25T01:35:33.165 INFO:scrape:Failure: Command failed on smithi036 with status 1: 'sudo ceph --cluster ceph osd crush tunables default' 2021-04-25T01:35:33.165 INFO:scrape:2 jobs: ['6067451', '6067314'] 2021-04-25T01:35:33.165 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'overrides/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-04-25T01:35:33.165 INFO:scrape:suites union: ['1-client', '2-upgrade', '3-verify}', 'clusters/1-mds-2-client-micro', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'distro~HEAD/{ubuntu_latest}', 'fs/upgrade/volumes/import-legacy/{bluestore-bitmap', 'fs/verify/{begin', 'mds', 'mon', 'mon-debug', 'mount/fuse', 'objectstore-ec/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'pg-warn', 'ranks/3', 'session_timeout', 'tasks/dbench', 'tasks/{0-nautilus', 'ubuntu_18.04}', 'validater/valgrind}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-04-25T01:35:33.166 INFO:scrape: 2021-04-25T01:35:33.166 INFO:scrape:Failure: Test failure: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2021-04-25T01:35:33.166 INFO:scrape:2 jobs: ['6067289', '6067487'] 2021-04-25T01:35:33.166 INFO:scrape:suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{frag_enable', 'tasks/mds-full}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-04-25T01:35:33.166 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/mds-full}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-04-25T01:35:33.166 INFO:scrape: 2021-04-25T01:35:33.166 INFO:scrape:Failure: Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_ino_release_cb' 2021-04-25T01:35:33.166 INFO:scrape:1 jobs: ['6067268'] 2021-04-25T01:35:33.167 INFO:scrape:suites: ['clusters/small-cluster', 'conf/{client', 'fs/bugs/client_trim_caps/{begin', 'mds', 'mon', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/trim-i22073}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-04-25T01:35:33.167 INFO:scrape: 2021-04-25T01:35:33.167 INFO:scrape:Failure: Test failure: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2021-04-25T01:35:33.167 INFO:scrape:1 jobs: ['6067342'] 2021-04-25T01:35:33.167 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/snap-schedule}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-04-25T01:35:33.167 INFO:scrape: 2021-04-25T01:35:33.167 INFO:scrape:Failure: The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} 2021-04-25T01:35:33.167 INFO:scrape:1 jobs: ['6067312'] 2021-04-25T01:35:33.167 INFO:scrape:suites: ['clusters/1a5s-mds-1c-client-3node', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/workload/{begin', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{frag_enable', 'ranks/5', 'scrub/yes', 'session_timeout', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/iogen}}'] 2021-04-25T01:35:33.167 INFO:scrape: 2021-04-25T01:35:33.168 INFO:scrape:Failure: Command failed (workunit test suites/pjd.sh) on smithi040 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=80c1bb7e30aa853a28409be8101c5a37d1e35fd3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' 2021-04-25T01:35:33.168 INFO:scrape:1 jobs: ['6067334'] 2021-04-25T01:35:33.168 INFO:scrape:suites: ['2-workunit/suites/pjd}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/thrash/workloads/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/mon', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-04-25T01:35:33.168 INFO:scrape: 2021-04-25T01:35:33.168 INFO:scrape:Failure: Command failed (workunit test fs/quota/quota.sh) on smithi139 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=80c1bb7e30aa853a28409be8101c5a37d1e35fd3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/quota/quota.sh' 2021-04-25T01:35:33.168 INFO:scrape:1 jobs: ['6067377'] 2021-04-25T01:35:33.168 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'tasks/workunit/quota}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-04-25T01:35:33.168 INFO:scrape: 2021-04-25T01:35:33.181 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/ failed: 30 dead: 0 running: 0 waiting: 0 queued: 0 passed: 230 Fail ================================================================= [6067268] fs/bugs/client_trim_caps/{begin clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/trim-i22073} ----------------------------------------------------------------- time: 00:08:43 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067268/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067268/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ace86ac84bfe4a92bf21fbccd3ad7250 Command crashed: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_ino_release_cb' [6067286] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/fs/misc}} ----------------------------------------------------------------- time: 00:44:42 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067286/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067286/ "2021-04-24T19:58:01.254307+0000 mds.c (mds.2) 4 : cluster [WRN] Scrub error on inode 0x10000000264 (/client.0/tmp/testdir/dir1/dir2) see mds.c log and `damage ls` output for details" in cluster log [6067289] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-full} ----------------------------------------------------------------- time: 00:17:19 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067289/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067289/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7b76fcd39a9f478c9280eabbd5567b43 Test failure: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) [6067301] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:15:09 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067301/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067301/ "2021-04-24T20:02:26.924844+0000 mds.b (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000012df (/client.0/tmp/fsstress/ltp- full-20091231/testcases/open_posix_testsuite) see mds.b log and `damage ls` output for details" in cluster log [6067312] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/iogen}} ----------------------------------------------------------------- time: 00:49:30 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067312/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067312/ The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} [6067314] fs/upgrade/volumes/import-legacy/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} ----------------------------------------------------------------- time: 01:10:50 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067314/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067314/ Command failed on smithi109 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' [6067329] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} ----------------------------------------------------------------- time: 00:16:15 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067329/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067329/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6cf2bfbf234247fc9856f936d625680b Test failure: test_delete_object (tasks.cephfs.test_volume_client.TestVolumeClient) [6067333] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} ----------------------------------------------------------------- time: 00:37:25 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067333/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067333/ "2021-04-24T20:30:55.748562+0000 mds.d (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000001fa (/client.0/tmp/blogbench-1.0/man) see mds.d log and `damage ls` output for details" in cluster log [6067334] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:18:06 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067334/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067334/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ae22a505ec1e45ad9c04fabed3f7bbc8 Command failed (workunit test suites/pjd.sh) on smithi040 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=80c1bb7e30aa853a28409be8101c5a37d1e35fd3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [6067338] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/ffsb}} ----------------------------------------------------------------- time: 01:11:25 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067338/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067338/ "2021-04-24T20:29:22.774041+0000 mds.c (mds.0) 14 : cluster [WRN] Scrub error on inode 0x10000000348 (/client.0/tmp/tmp) see mds.c log and `damage ls` output for details" in cluster log [6067342] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snap-schedule} ----------------------------------------------------------------- time: 00:13:59 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067342/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067342/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6972d593925d460b8f44c5b982d23140 Test failure: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) [6067366] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} distro~HEAD/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/dbench validater/valgrind} ----------------------------------------------------------------- time: 00:19:33 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067366/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067366/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ec5d51e39cfe42d28a1be0d382b9a044 Command failed on smithi019 with status 1: "sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust- ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/client.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' --id 0 /home/ubuntu/cephtest/mnt.0" [6067368] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/fs/misc}} ----------------------------------------------------------------- time: 03:23:03 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067368/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067368/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d4baf8ac12df4740bc7bd843446e33a4 Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi092 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=80c1bb7e30aa853a28409be8101c5a37d1e35fd3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/misc/multiple_rsync.sh' [6067375] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} ----------------------------------------------------------------- time: 00:19:15 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067375/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067375/ "2021-04-24T20:54:21.610533+0000 mds.f (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000001fe (/client.0/tmp/blogbench-1.0/src) see mds.f log and `damage ls` output for details" in cluster log [6067377] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/quota} ----------------------------------------------------------------- time: 00:13:53 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067377/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067377/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3e1a3933ee424fc983698445e9afa121 Command failed (workunit test fs/quota/quota.sh) on smithi139 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=80c1bb7e30aa853a28409be8101c5a37d1e35fd3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/quota/quota.sh' [6067386] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:14:17 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067386/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067386/ "2021-04-24T21:03:57.589973+0000 mds.f (mds.0) 14 : cluster [WRN] Scrub error on inode 0x10000000c23 (/client.0/tmp/fsstress/ltp- full-20091231/testcases/open_hpi_testsuite) see mds.f log and `damage ls` output for details" in cluster log [6067401] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:15:47 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067401/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067401/ "2021-04-24T21:16:25.539120+0000 mds.d (mds.1) 11 : cluster [WRN] Scrub error on inode 0x10000000244 (/client.0/tmp/pjd- fstest-20090130-RC/tests/rmdir) see mds.d log and `damage ls` output for details" in cluster log [6067412] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/fs/misc}} ----------------------------------------------------------------- time: 00:48:11 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067412/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067412/ "2021-04-24T21:23:46.321860+0000 mds.b (mds.2) 4 : cluster [WRN] Scrub error on inode 0x100000001ff (/client.0/tmp/testdir/dir1/dir2) see mds.b log and `damage ls` output for details" in cluster log [6067423] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:49:44 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067423/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067423/ "2021-04-24T21:30:37.375632+0000 mds.e (mds.0) 19 : cluster [WRN] Scrub error on inode 0x10000000347 (/client.0/tmp/tmp) see mds.e log and `damage ls` output for details" in cluster log [6067428] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:16:45 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067428/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067428/ "2021-04-24T21:37:31.797621+0000 mds.a (mds.0) 14 : cluster [WRN] Scrub error on inode 0x1000000140a (/client.0/tmp/fsstress/ltp- full-20091231/testcases/open_posix_testsuite/conformance/interfaces) see mds.a log and `damage ls` output for details" in cluster log [6067443] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/damage} ----------------------------------------------------------------- time: 00:20:01 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067443/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067443/ sentry: https://sentry.ceph.com/organizations/ceph/?query=83ee4269b67140a19057d7f340ded5f7 Test failure: test_object_deletion (tasks.cephfs.test_damage.TestDamage) [6067445] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:12:14 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067445/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067445/ "2021-04-24T21:43:08.234625+0000 mds.b (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000002cd (/client.0/tmp/tmp) see mds.b log and `damage ls` output for details" in cluster log [6067451] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} distro~HEAD/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/dbench validater/valgrind} ----------------------------------------------------------------- time: 01:54:08 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067451/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067451/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3f229b7342834d768cba281d7b1bc010 Command failed on smithi036 with status 1: 'sudo ceph --cluster ceph osd crush tunables default' [6067454] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/fs/misc}} ----------------------------------------------------------------- time: 00:39:55 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067454/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067454/ "2021-04-23T08:38:56.307298+0000 mds.c (mds.0) 109 : cluster [WRN] Scrub error on inode 0x1000000f4b9 (/client.0/tmp/payload.1/multiple_rsync_payloa d.152605/modules/4.18.0-240.el8.x86_64/kernel/drivers/hwmon) see mds.c log and `damage ls` output for details" in cluster log [6067459] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} ----------------------------------------------------------------- time: 01:06:32 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067459/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067459/ sentry: https://sentry.ceph.com/organizations/ceph/?query=37b380c60a0a42dd8e1110a9a1d4b76b Command failed on smithi018 with status 1: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp' [6067487] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-full} ----------------------------------------------------------------- time: 00:20:18 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067487/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067487/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4e757355e378483a99f95e06ffc801aa Test failure: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) [6067495] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/fs/misc}} ----------------------------------------------------------------- time: 03:20:46 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067495/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067495/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5a955da4aa6f4a16968809d1af6adf5a Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi032 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=80c1bb7e30aa853a28409be8101c5a37d1e35fd3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/misc/multiple_rsync.sh' [6067500] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} ----------------------------------------------------------------- time: 00:49:32 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067500/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067500/ "2021-04-24T22:25:57.073110+0000 mds.f (mds.0) 14 : cluster [WRN] Scrub error on inode 0x100000001fe (/client.0/tmp/blogbench-1.0/src) see mds.f log and `damage ls` output for details" in cluster log [6067513] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} ----------------------------------------------------------------- time: 00:15:16 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067513/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067513/ sentry: https://sentry.ceph.com/organizations/ceph/?query=325dc4ad28db41ab8880cf8917195827 Test failure: test_delete_object (tasks.cephfs.test_volume_client.TestVolumeClient) [6067514] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:19:48 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067514/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067514/ "2021-04-24T22:30:29.854437+0000 mds.d (mds.0) 14 : cluster [WRN] Scrub error on inode 0x10000002dfe (/client.0/tmp/fsstress/ltp- full-20091231/testcases/realtime) see mds.d log and `damage ls` output for details" in cluster log Pass ================================================================= [6067265] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:16:13 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067265/ [6067266] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/pjd}} time: 00:14:49 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067266/ [6067267] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:13:02 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067267/ [6067269] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/centos_latest} time: 00:43:03 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067269/ [6067270] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush} time: 00:13:29 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067270/ [6067271] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client} time: 00:12:57 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067271/ [6067272] fs/mirror/{begin cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{ubuntu_latest} tasks/mirror} time: 01:03:25 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067272/ [6067273] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:31:03 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067273/ [6067274] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:18:57 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067274/ [6067275] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 00:59:05 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067275/ [6067276] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:13:40 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067276/ [6067277] fs/shell/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cephfs-shell} time: 00:33:02 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067277/ [6067278] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:55:24 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067278/ [6067279] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} time: 00:42:40 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067279/ [6067280] fs/top/{begin cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/whitelist_health supported-random-distros$/{rhel_8} tasks/fstop} time: 00:12:14 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067280/ [6067281] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:19:55 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067281/ [6067282] fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}} time: 00:27:41 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067282/ [6067283] fs/valgrind/{begin centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror}} time: 01:20:28 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067283/ [6067284] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} distro~HEAD/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/dbench validater/valgrind} time: 00:55:48 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067284/ [6067285] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/direct_io}} time: 00:14:19 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067285/ [6067287] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 01:07:31 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067287/ [6067288] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/kernel_untar_build}} time: 00:33:48 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067288/ [6067290] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:32:57 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067290/ [6067291] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} time: 00:24:25 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067291/ [6067292] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:28:02 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067292/ [6067293] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:21:17 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067293/ [6067294] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/suites/dbench}} time: 00:43:14 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067294/ [6067295] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds_creation_retry} time: 00:12:26 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067295/ [6067296] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/ffsb}} time: 00:54:12 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067296/ [6067297] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:13:04 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067297/ [6067298] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/fs/norstats}} time: 00:13:06 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067298/ [6067299] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/metrics} time: 00:22:19 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067299/ [6067300] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:14:57 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067300/ [6067302] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:12:17 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067302/ [6067303] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/suites/fsx}} time: 00:18:26 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067303/ [6067304] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/multimds_misc} time: 00:20:25 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067304/ [6067305] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/fsync-tester}} time: 00:14:31 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067305/ [6067306] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:25:27 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067306/ [6067307] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc}} time: 00:21:32 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067307/ [6067308] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:13:51 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067308/ [6067309] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/openfiletable} time: 00:16:43 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067309/ [6067310] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:11:51 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067310/ [6067311] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 01:19:34 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067311/ [6067313] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:31:27 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067313/ [6067315] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/suites/iozone}} time: 00:14:38 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067315/ [6067316] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm} time: 00:10:06 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067316/ [6067317] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/pjd}} time: 00:15:56 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067317/ [6067318] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:16:59 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067318/ [6067319] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 01:30:16 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067319/ [6067320] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:09:28 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067320/ [6067321] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:27:36 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067321/ [6067322] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} distro~HEAD/{rhel_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/fsstress validater/lockdep} time: 00:21:35 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067322/ [6067323] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/direct_io}} time: 00:11:53 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067323/ [6067324] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:09:59 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067324/ [6067325] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/quota} time: 00:14:00 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067325/ [6067326] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/fs/misc}} time: 00:54:24 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067326/ [6067327] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:35:24 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067327/ [6067328] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} time: 00:20:40 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067328/ [6067330] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:20:04 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067330/ [6067331] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/kernel_untar_build}} time: 00:28:11 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067331/ [6067332] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/scrub} time: 00:25:12 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067332/ [6067335] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/suites/dbench}} time: 00:51:29 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067335/ [6067336] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} time: 00:15:36 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067336/ [6067337] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/sessionmap} time: 00:14:26 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067337/ [6067339] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:03:46 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067339/ [6067340] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/fs/norstats}} time: 00:15:15 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067340/ [6067341] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:13:20 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067341/ [6067343] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:11:00 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067343/ [6067344] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/fsstress}} time: 00:20:02 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067344/ [6067345] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:32:07 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067345/ [6067346] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/suites/fsx}} time: 00:22:20 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067346/ [6067347] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:18:00 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067347/ [6067348] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/fsync-tester}} time: 00:20:37 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067348/ [6067349] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snapshots} time: 00:21:53 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067349/ [6067350] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:13:06 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067350/ [6067351] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:18:59 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067351/ [6067352] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc}} time: 00:24:36 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067352/ [6067353] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/iogen}} time: 00:50:18 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067353/ [6067354] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/strays} time: 00:28:45 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067354/ [6067355] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:14:51 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067355/ [6067356] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/suites/iozone}} time: 00:15:50 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067356/ [6067357] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:12:19 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067357/ [6067358] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/pjd}} time: 00:12:29 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067358/ [6067359] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/test_journal_migration} time: 00:18:03 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067359/ [6067360] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:46:19 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067360/ [6067361] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs} time: 00:14:43 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067361/ [6067362] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:39:04 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067362/ [6067363] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} time: 00:21:52 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067363/ [6067364] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 01:25:40 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067364/ [6067365] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} time: 00:13:54 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067365/ [6067367] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/direct_io}} time: 00:14:05 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067367/ [6067369] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 01:01:52 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067369/ [6067370] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/truncate_delay} time: 00:10:03 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067370/ [6067371] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/kernel_untar_build}} time: 00:46:36 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067371/ [6067372] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:04:16 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067372/ [6067373] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} time: 00:54:09 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067373/ [6067374] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:33:14 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067374/ [6067376] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:14:31 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067376/ [6067378] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:10:41 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067378/ [6067379] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:30:43 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067379/ [6067380] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/suites/dbench}} time: 01:26:50 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067380/ [6067381] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/ffsb}} time: 00:21:50 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067381/ [6067382] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:10:10 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067382/ [6067383] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/acls} time: 00:14:38 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067383/ [6067384] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/fs/norstats}} time: 00:13:48 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067384/ [6067385] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:20:43 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067385/ [6067387] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/admin} time: 00:39:00 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067387/ [6067388] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:16:16 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067388/ [6067389] fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:18:34 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067389/ [6067390] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/suites/fsx}} time: 00:23:33 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067390/ [6067391] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/fsync-tester}} time: 00:14:22 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067391/ [6067392] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/ffsb}} time: 00:21:12 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067392/ [6067393] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/alternate-pool} time: 00:11:40 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067393/ [6067394] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc}} time: 00:21:36 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067394/ [6067395] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 01:13:10 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067395/ [6067396] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/iogen}} time: 00:45:39 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067396/ [6067397] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:38:03 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067397/ [6067398] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/asok_dump_tree} time: 00:23:24 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067398/ [6067399] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:30:15 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067399/ [6067400] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/suites/iozone}} time: 00:16:27 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067400/ [6067402] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:21:00 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067402/ [6067403] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:14:35 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067403/ [6067404] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/auto-repair} time: 00:13:52 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067404/ [6067405] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 02:25:15 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067405/ [6067406] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:18:55 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067406/ [6067407] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:31:31 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067407/ [6067408] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:13:49 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067408/ [6067409] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} distro~HEAD/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/fsstress validater/lockdep} time: 00:18:41 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067409/ [6067410] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/direct_io}} time: 00:13:31 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067410/ [6067411] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:13:01 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067411/ [6067413] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:14:21 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067413/ [6067414] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/kernel_untar_build}} time: 00:40:32 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067414/ [6067415] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/backtrace} time: 00:13:47 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067415/ [6067416] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/blogbench}} time: 00:21:16 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067416/ [6067417] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:11:56 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067417/ [6067418] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:41:17 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067418/ [6067419] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} time: 01:13:24 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067419/ [6067420] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/suites/dbench}} time: 01:08:18 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067420/ [6067421] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cap-flush} time: 00:11:20 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067421/ [6067422] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/iozone}} time: 00:17:43 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067422/ [6067424] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 00:53:35 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067424/ [6067425] fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}} time: 00:27:17 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067425/ [6067426] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/fs/norstats}} time: 00:16:05 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067426/ [6067427] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-limits} time: 00:21:36 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067427/ [6067429] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:34:57 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067429/ [6067430] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/suites/fsx}} time: 00:29:37 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067430/ [6067431] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-readahead} time: 00:13:11 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067431/ [6067432] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:16:58 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067432/ [6067433] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/fsync-tester}} time: 00:13:01 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067433/ [6067434] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:19:31 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067434/ [6067435] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:10:17 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067435/ [6067436] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc}} time: 00:23:26 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067436/ [6067437] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:15:15 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067437/ [6067438] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery} time: 00:35:24 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067438/ [6067439] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:10:12 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067439/ [6067440] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/iogen}} time: 00:24:23 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067440/ [6067441] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:22:11 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067441/ [6067442] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/suites/iozone}} time: 00:16:59 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067442/ [6067444] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:16:07 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067444/ [6067446] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs_python} time: 00:12:42 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067446/ [6067447] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:35:34 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067447/ [6067448] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file} time: 00:08:59 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067448/ [6067449] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth} time: 00:30:24 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067449/ [6067450] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} time: 01:06:39 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067450/ [6067452] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/direct_io}} time: 00:16:10 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067452/ [6067453] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/data-scan} time: 00:27:40 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067453/ [6067455] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 00:49:31 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067455/ [6067456] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/kernel_untar_build}} time: 00:34:45 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067456/ [6067457] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:36:06 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067457/ [6067458] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/exports} time: 00:41:29 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067458/ [6067460] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:15:48 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067460/ [6067461] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:19:39 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067461/ [6067462] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/suites/dbench}} time: 00:49:46 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067462/ [6067463] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub} time: 00:17:45 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067463/ [6067464] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/ffsb}} time: 00:57:44 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067464/ [6067465] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:14:59 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067465/ [6067466] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:13:48 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067466/ [6067467] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:13:02 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067467/ [6067468] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/fs/norstats}} time: 00:16:06 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067468/ [6067469] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:14:21 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067469/ [6067470] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:10:42 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067470/ [6067471] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/fragment} time: 00:17:24 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067471/ [6067472] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:11:11 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067472/ [6067473] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/fsstress}} time: 00:15:52 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067473/ [6067474] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:15:17 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067474/ [6067475] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/suites/fsx}} time: 00:37:51 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067475/ [6067476] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:22:06 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067476/ [6067477] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes tasks/{0-check-counter workunit/suites/fsync-tester}} time: 00:15:31 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067477/ [6067478] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/journal-repair} time: 00:19:40 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067478/ [6067479] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:20:10 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067479/ [6067480] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc}} time: 00:23:21 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067480/ [6067481] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 02:05:16 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067481/ [6067482] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/iogen}} time: 00:54:11 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067482/ [6067483] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush} time: 00:13:20 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067483/ [6067484] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:49:46 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067484/ [6067485] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/suites/iozone}} time: 00:15:14 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067485/ [6067486] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/pjd}} time: 00:15:39 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067486/ [6067488] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:17:07 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067488/ [6067489] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone} time: 01:48:41 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067489/ [6067490] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/mdtest} time: 00:11:49 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067490/ [6067491] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover} time: 01:01:39 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067491/ [6067492] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} distro~HEAD/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/fsstress validater/lockdep} time: 00:18:50 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067492/ [6067493] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/direct_io}} time: 00:13:54 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067493/ [6067494] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:11:48 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067494/ [6067496] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds_creation_retry} time: 00:10:09 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067496/ [6067497] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:17:36 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067497/ [6067498] fs/upgrade/featureful_client/old_client/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{frag_enable multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:19:49 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067498/ [6067499] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/kernel_untar_build}} time: 01:21:27 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067499/ [6067501] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:18:05 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067501/ [6067502] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:18:45 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067502/ [6067503] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/metrics} time: 00:21:30 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067503/ [6067504] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:14:32 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067504/ [6067505] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/suites/dbench}} time: 01:07:50 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067505/ [6067506] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag_enable multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} time: 00:15:41 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067506/ [6067507] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/ffsb}} time: 00:55:55 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067507/ [6067508] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/multimds_misc} time: 00:19:17 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067508/ [6067509] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 00:43:33 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067509/ [6067510] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no tasks/{0-check-counter workunit/fs/norstats}} time: 00:16:03 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067510/ [6067511] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:07:38 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067511/ [6067512] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} time: 00:22:55 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067512/ [6067515] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:42:56 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067515/ [6067516] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/openfiletable} time: 00:19:15 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067516/ [6067517] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no tasks/{0-check-counter workunit/suites/fsx}} time: 00:26:32 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067517/ [6067518] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:14:12 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067518/ [6067519] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes tasks/{0-check-counter workunit/suites/fsync-tester}} time: 00:13:39 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067519/ [6067520] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm} time: 00:11:53 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067520/ [6067521] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:13:42 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067521/ [6067522] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no tasks/{0-check-counter workunit/fs/test_o_trunc}} time: 00:20:15 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067522/ [6067523] fs/workload/{begin clusters/1a5s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes tasks/{0-check-counter workunit/suites/iogen}} time: 00:59:23 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067523/ [6067524] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:15:24 info: http://pulpito.ceph.com/teuthology-2021-04-23_04:17:03-fs-pacific-distro-basic-smithi/6067524/